Build a BGP ISP Lab with FRRouting - Part 1

In the world of network engineering, hands-on experience with BGP and ISP-grade routing is invaluable. This comprehensive tutorial guides you through building a complete, functional ISP environment on your laptop using lightweight Linux VMs and FRRouting.

Lab Overview

We'll create a 5-router topology that mirrors a real ISP network:

• AS 65001: Your ISP core network (3 routers)
• AS 65002: Upstream provider
• AS 65003: Customer network

Prerequisites and Requirements

Hardware Requirements

Software Requirements

• A hypervisor of your choice
• Ubuntu Server 24 LTS Noble (x86)
• SSH client
• Basic knowledge of Linux CLI and networking

Network Architecture

Topology Design

                      VM2 (Provider)
                   10.2.2.2/32 | AS 65002
                               |
                          eBGP |
                               |
                      VM1 (Route Reflector)
                   10.1.1.1/32 | AS 65001
                               |
                 +-------------+-------------+
                 |                           |
            iBGP |                      iBGP |
                 |                           |
         VM3 (ISP Router)            VM4 (ISP Router)
      10.3.3.3/32 | AS 65001      10.4.4.4/32 | AS 65001


                      VM1 (Route Reflector)
                   10.1.1.1/32 | AS 65001
                               |
                          eBGP |
                               |
                     VM5 (Customer)
                   10.5.5.5/32 | AS 65003

BGP Relationships:
------------------
• VM2 ←→ VM1: eBGP (External BGP between different AS)
• VM1 ←→ VM3: iBGP (Internal BGP within AS 65001)
• VM1 ←→ VM4: iBGP (Internal BGP within AS 65001)
• VM1 ←→ VM5: eBGP (External BGP between different AS)

IP Addressing Scheme

Step-by-Step VM Setup

1 Create Base VM Template

VM Configuration

Create a base VM with the following specifications:

• Name: ubuntu-base
• OS: Linux, Ubuntu (64-bit)
• Memory: 1024 MB
• Disk: 10GB (dynamic allocation)

Install Ubuntu Server

# Download Ubuntu Server 24.04 LTS Noble ISO (x86)
wget https://releases.ubuntu.com/24.04.1/ubuntu-24.04.1-live-server-amd64.iso

During installation:

• Language: English
• Network: DHCP (we'll configure static later)
• Storage: Use entire disk
• Username: netadmin
• Install OpenSSH server: Yes
• No additional packages

2 Prepare Base Template

After installation, log in and run these commands:

# Update system
sudo apt update && sudo apt upgrade -y

# Install essential tools
sudo apt install -y net-tools vim curl wget htop tcpdump

# Disable cloud-init (causes issues with cloning)
sudo touch /etc/cloud/cloud-init.disabled

# Clear machine ID for cloning
sudo truncate -s 0 /etc/machine-id
sudo rm /var/lib/dbus/machine-id
sudo ln -s /etc/machine-id /var/lib/dbus/machine-id

# Shutdown for cloning
sudo shutdown -h now

3 Spin Up 5 VMs

Clone the base VM to create 5 virtual machines:

• vm1 - Route Reflector (AS 65001)
• vm2 - Provider (AS 65002)
• vm3 - ISP Router (AS 65001)
• vm4 - ISP Router (AS 65001)
• vm5 - Customer (AS 65003)

Network Configuration

1 Configure Static IPs

On each VM, configure the network interface. Example for VM1:

# Edit netplan configuration
sudo vim /etc/netplan/00-installer-config.yaml

VM1 Netplan Configuration

network:
  version: 2
  ethernets:
    enp0s3:  # Adjust interface name as needed
      addresses:
        - 192.168.139.115/24
      routes:
        - to: default
          via: 192.168.139.1
      nameservers:
        addresses:
          - 8.8.8.8
          - 8.8.4.4

# Apply configuration
sudo netplan apply

# Verify
ip addr show
ping 8.8.8.8

2 Set Hostnames

On each VM, set the appropriate hostname:

# For VM1 (repeat with vm2, vm3, vm4, vm5 on respective VMs)
sudo hostnamectl set-hostname vm1
sudo bash -c 'echo "127.0.1.1 vm1" >> /etc/hosts'

# Verify
hostname

3 Configure /etc/hosts

Add all VMs to /etc/hosts on each machine:

sudo bash -c 'cat >> /etc/hosts << EOF
192.168.139.115 vm1
192.168.139.145 vm2
192.168.139.225 vm3
192.168.139.147 vm4
192.168.139.17  vm5
EOF'

4 Test Connectivity

# From each VM, test connectivity
for i in vm1 vm2 vm3 vm4 vm5; do
    ping -c 1 $i && echo "$i is reachable" || echo "$i is NOT reachable"
done

FRRouting Installation

1 Install FRRouting Package

Run on all VMs:

# Update package list
sudo apt update

# Install FRRouting
sudo apt install -y frr frr-pythontools

# Verify installation
vtysh -v

2 Enable Required Daemons

# Edit daemon configuration
sudo vim /etc/frr/daemons

# Enable these daemons (set to yes):
# zebra=yes
# bgpd=yes
# staticd=yes

# Or use sed commands:
sudo sed -i 's/^zebra=.*/zebra=yes/' /etc/frr/daemons
sudo sed -i 's/^bgpd=.*/bgpd=yes/' /etc/frr/daemons
sudo sed -i 's/^staticd=.*/staticd=yes/' /etc/frr/daemons

3 Configure Loopback Interfaces

Add loopback IPs on each VM:

# VM1
sudo ip addr add 10.1.1.1/32 dev lo

# VM2
sudo ip addr add 10.2.2.2/32 dev lo

# VM3
sudo ip addr add 10.3.3.3/32 dev lo

# VM4
sudo ip addr add 10.4.4.4/32 dev lo

# VM5
sudo ip addr add 10.5.5.5/32 dev lo

# Make persistent (add to /etc/network/interfaces or netplan)

BGP Configuration

Automated Setup Script

Create and run this script on each VM:

#!/usr/bin/env bash
# Save as setup-bgp.sh
set -euo pipefail

echo "=== BGP lab setup starting on host: $(hostname) ==="

HOSTNAME=$(hostname)

# LAN IPs
VM1_IP="192.168.139.115"
VM2_IP="192.168.139.145"
VM3_IP="192.168.139.225"
VM4_IP="192.168.139.147"
VM5_IP="192.168.139.17"

case "$HOSTNAME" in
  vm1)
    # Route Reflector Configuration
    cat << 'EOF' | sudo tee /etc/frr/frr.conf
hostname vm1
password zebra
!
router bgp 65001
 bgp router-id 1.1.1.1
 neighbor 192.168.139.225 remote-as 65001
 neighbor 192.168.139.225 route-reflector-client
 neighbor 192.168.139.147 remote-as 65001
 neighbor 192.168.139.147 route-reflector-client
 neighbor 192.168.139.145 remote-as 65002
 neighbor 192.168.139.17 remote-as 65003
 !
 address-family ipv4 unicast
  network 10.1.1.1/32
  neighbor 192.168.139.225 activate
  neighbor 192.168.139.147 activate
  neighbor 192.168.139.145 activate
  neighbor 192.168.139.17 activate
 exit-address-family
!
EOF
    ;;

  vm2)
    # Upstream Provider Configuration
    cat << 'EOF' | sudo tee /etc/frr/frr.conf
hostname vm2
password zebra
!
router bgp 65002
 bgp router-id 2.2.2.2
 neighbor 192.168.139.115 remote-as 65001
 !
 address-family ipv4 unicast
  network 10.2.2.2/32
  neighbor 192.168.139.115 activate
 exit-address-family
!
EOF
    ;;

  vm3)
    # iBGP Client 1 Configuration
    cat << 'EOF' | sudo tee /etc/frr/frr.conf
hostname vm3
password zebra
!
router bgp 65001
 bgp router-id 3.3.3.3
 neighbor 192.168.139.115 remote-as 65001
 !
 address-family ipv4 unicast
  network 10.3.3.3/32
  neighbor 192.168.139.115 activate
 exit-address-family
!
EOF
    ;;

  vm4)
    # iBGP Client 2 Configuration
    cat << 'EOF' | sudo tee /etc/frr/frr.conf
hostname vm4
password zebra
!
router bgp 65001
 bgp router-id 4.4.4.4
 neighbor 192.168.139.115 remote-as 65001
 !
 address-family ipv4 unicast
  network 10.4.4.4/32
  neighbor 192.168.139.115 activate
 exit-address-family
!
EOF
    ;;

  vm5)
    # Customer Configuration
    cat << 'EOF' | sudo tee /etc/frr/frr.conf
hostname vm5
password zebra
!
router bgp 65003
 bgp router-id 5.5.5.5
 neighbor 192.168.139.115 remote-as 65001
 !
 address-family ipv4 unicast
  network 10.5.5.5/32
  neighbor 192.168.139.115 activate
 exit-address-family
!
EOF
    ;;

  *)
    echo "Unknown hostname: $HOSTNAME"
    exit 1
    ;;
esac

# Restart FRR
sudo systemctl restart frr
echo "=== BGP setup completed on $(hostname) ==="

# Make script executable and run
chmod +x setup-bgp.sh
./setup-bgp.sh

Verification and Testing

1 Verify BGP Sessions

# Check BGP summary
sudo vtysh -c "show ip bgp summary"

# Expected output shows established sessions (state = number of prefixes)

Expected BGP Summary Output (VM1)

IPv4 Unicast Summary:
BGP router identifier 1.1.1.1, local AS number 65001
Neighbor        V         AS   MsgRcvd   MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd
192.168.139.145 4      65002        12        15        0    0    0 00:05:23            1
192.168.139.225 4      65001        10        12        0    0    0 00:04:15            1
192.168.139.147 4      65001        10        12        0    0    0 00:04:10            1
192.168.139.17  4      65003        11        14        0    0    0 00:03:45            1

2 View BGP Routing Table

# Show all BGP routes
sudo vtysh -c "show ip bgp"

# Show specific route details
sudo vtysh -c "show ip bgp 10.5.5.5/32"

3 Test End-to-End Connectivity

# From VM2 (Provider), ping VM5 (Customer) loopback
ping -c 4 10.5.5.5

# Traceroute to see path
traceroute 10.5.5.5

Advanced BGP Configuration

Implementing Route Filtering

Implement a route filter on VM1 to prevent VM5's routes from reaching VM2:

# On VM1
sudo vtysh
configure terminal

ip prefix-list BLOCK_VM5 seq 10 deny 10.5.5.5/32
ip prefix-list BLOCK_VM5 seq 20 permit 0.0.0.0/0 le 32

router bgp 65001
  neighbor 192.168.139.145 prefix-list BLOCK_VM5 out

end
write
clear ip bgp 192.168.139.145 soft out

Configuring AS Path Prepending

Make VM2 prefer routes through a longer path:

# On VM1
sudo vtysh
configure terminal

route-map PREPEND_TO_VM2 permit 10
  set as-path prepend 65001 65001

router bgp 65001
  neighbor 192.168.139.145 route-map PREPEND_TO_VM2 out

end
write
clear ip bgp 192.168.139.145 soft out

Setting Local Preference

Set local preference to control outbound traffic:

# On VM1
sudo vtysh
configure terminal

route-map SET_LOCALPREF permit 10
  set local-preference 200

router bgp 65001
  neighbor 192.168.139.145 route-map SET_LOCALPREF in

end
write
clear ip bgp 192.168.139.145 soft in

Troubleshooting Guide

Diagnostic Commands

# BGP Diagnostics
sudo vtysh -c "show ip bgp summary"
sudo vtysh -c "show ip bgp"
sudo vtysh -c "show ip bgp neighbors"

# System Diagnostics
sudo systemctl status frr
sudo journalctl -u frr -f
sudo tcpdump -i any -n port 179

Conclusion

Next Steps

1. Add IPv6 support (MP-BGP)
2. Implement MPLS and VRFs
3. Configure BGP FlowSpec for DDoS mitigation
4. Add OSPF/IS-IS for IGP
5. Integrate with SDN controllers