Skip to main content

Network Configuration on Each Node

Two Bridges, Three Nodes

You're adding vmbr_priv and vmbr_pub to each node. The vmbr0 bridge from the first guide stays exactly as it is — these are additions. The config is identical across all three nodes except for the IP on the private bridge, which is unique per node.

You'll need the two VLAN IDs from the previous chapter. They appear as <PRIV_VLAN> and <PUB_VLAN> below — substitute your actual values everywhere they appear.

MTU 1400 — Why It Matters and Why People Skip It

Every unexplained connection problem I've run into on Hetzner vSwitches has come down to MTU. The symptoms are the worst kind — small packets work fine, SSH works fine, a quick curl works fine, but large file transfers stall or a VPN never fully handshakes. It looks like a routing problem or a firewall rule. It isn't.

Hetzner vSwitches use VXLAN internally. VXLAN adds 50 bytes of header overhead per frame. If your bridges and VM interfaces sit at the standard 1500-byte MTU, frames near that ceiling get silently fragmented. Set everything to 1400 and that whole class of problem disappears. This applies to the bridges here, PfSense's interfaces in Chapter 5, and every VM NIC in Chapter 6 — anything that touches a vSwitch bridge.

Step 1 — Enable VLAN Support on Each Node

apt-get install -y vlan
echo "8021q" >> /etc/modules
modprobe 8021q

Step 2 — Edit /etc/network/interfaces

Append the blocks below to the existing file on each node. If your main NIC is eth0 rather than eno1, swap the name throughout (ip link show if you're unsure).

Node 1 — pve1

# Private vSwitch
auto eno1.<PRIV_VLAN>
iface eno1.<PRIV_VLAN> inet manual
    vlan-raw-device eno1

auto vmbr_priv
iface vmbr_priv inet static
    address 10.100.101.1/23
    bridge-ports eno1.<PRIV_VLAN>
    bridge-stp off
    bridge-fd 0
    mtu 1400

# Public vSwitch
auto eno1.<PUB_VLAN>
iface eno1.<PUB_VLAN> inet manual
    vlan-raw-device eno1

auto vmbr_pub
iface vmbr_pub inet manual
    bridge-ports eno1.<PUB_VLAN>
    bridge-stp off
    bridge-fd 0
    mtu 1400

Node 2 — pve2 (only the address changes)

auto eno1.<PRIV_VLAN>
iface eno1.<PRIV_VLAN> inet manual
    vlan-raw-device eno1

auto vmbr_priv
iface vmbr_priv inet static
    address 10.100.101.2/23
    bridge-ports eno1.<PRIV_VLAN>
    bridge-stp off
    bridge-fd 0
    mtu 1400

auto eno1.<PUB_VLAN>
iface eno1.<PUB_VLAN> inet manual
    vlan-raw-device eno1

auto vmbr_pub
iface vmbr_pub inet manual
    bridge-ports eno1.<PUB_VLAN>
    bridge-stp off
    bridge-fd 0
    mtu 1400

Node 3 — pve3

auto eno1.<PRIV_VLAN>
iface eno1.<PRIV_VLAN> inet manual
    vlan-raw-device eno1

auto vmbr_priv
iface vmbr_priv inet static
    address 10.100.101.3/23
    bridge-ports eno1.<PRIV_VLAN>
    bridge-stp off
    bridge-fd 0
    mtu 1400

auto eno1.<PUB_VLAN>
iface eno1.<PUB_VLAN> inet manual
    vlan-raw-device eno1

auto vmbr_pub
iface vmbr_pub inet manual
    bridge-ports eno1.<PUB_VLAN>
    bridge-stp off
    bridge-fd 0
    mtu 1400

Step 3 — Apply and Verify

ifreload -a

Check both bridges came up:

ip addr show vmbr_priv
ip addr show vmbr_pub

vmbr_priv should show its 10.100.101.x/23 address. vmbr_pub should be UP with no IP — that's correct, the node intentionally has no address on the public bridge.

Confirm MTU took effect:

Both should say mtu 1400.

Step 4 — Ping Between Nodes Before Clustering

Don't skip this. From node1:

ping -c 4 10.100.101.2
ping -c 4 10.100.101.3

Both must respond before you touch pvecm. If Corosync starts before the private network is working, it falls back to binding on eth0 — silently, without an error. You end up with cluster heartbeat sharing your public management interface, and the only way you notice is when things go wrong under load. The pings take ten seconds. Do them.

If either ping fails: check the VLAN ID in your bridge config against what Robot shows for the private vSwitch, confirm both servers are listed as members, and verify lsmod | grep 8021q shows the module loaded.

No comments to display

Back to top