Skip to content
/ opnatt Public
forked from owenthewizard/opnatt

Use your own OPNsense router with AT&T Fiber.

3 stars 175 forks Branches Tags Activity
Star
Notifications You must be signed in to change notification settings

madeofstown/opnatt

BranchesTags
 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

102 Commits
bin
bin
 
 
img
img
 
 
wpa
wpa
 
 
README.md
README.md
 
 
U-VERSE_TV.md
U-VERSE_TV.md
 
 

Repository files navigation

Disclaimer

Bypassing the AT&T Residential Gateway does NOT allow for theft-of-service.

Service authentication is NOT performed by the AT&T Residential Gateway (likely the ONT instead)[citation needed]. This repository does NOT contain ANY code or any other content that is proprietary, patended, copyrighted, or otherwise belonging to AT&T, Arris, Motorola, or other companies related to the AT&T Residential Gateway.

About

This repository includes notes on using your own OPNsense hardware with AT&T Fiber. The method described here uses wpa_supplicant to authenticate via 802.1x (EAP-TLS), and either netgraph or a smart/managed switch to deal with VLAN ID 0. This method eliminates the need for the AT&T Residential Gateway entirely. The ONT is still used, and plugs directly into OPNsense.

Compatibility

This repository aims to support the latest stable version of OPNsense. Compatibility with older versions or other operating systems (pfSense, Linux, etc.) is not guaranteed. That said, netgraph is the only OS-specific part of the stack. Using a smart/managed switch instead of netgraph, eBPF (Linux), or another solution should work on other operating systems.

OPNsense

The latest OPNsense release this script was tested on is OPNsense 23.1.5_4-amd64.

GPON & XGS-PON

This type of bypass does not work in XGS-PON areas.

Switches

Manufacturer Model Working HW Version SW Version Notes Updated
D-Link DGS-1100-08 âś… B1 1.00.b031 @gpz1100 04/08/2023
D-Link DGS-1100-05v2 âś… A1 1.00.003 @topsecretsauce 04/12/2023
TP-Link TL-SG108E âś… 5.0 1.0.0 Build 20191021 Rel.53360 @owenthewizard 04/13/2023
D-Link DGS-1210-10 âś… F1 6.31.002 @gpz1100 04/14/2023
Netgear GS308T ❌ @gpz1100 04/08/2023
Cisco Catalyst 3750G âť”
Mikrotik RB4011 âť”
Ubiquiti USW-Flex-Mini ❌ @bigjohns97 04/14/2023

Background

While many AT&T Residential Gateways offer something called IP Passthrough, it does not provide the same advantages of a true bridge mode. For example, the NAT table is still managed by the gateway, which is limited to a measly 8192 sessions (although it becomes unstable at even 60% capacity)[citation needed].

The method described here will allow you to fully utilize your own router and fully bypass your Residential Gateway. It survives reboots, re-authentications, IPv6, and new DHCP leases.

How it Works

Before continuing to the setup, it's important to understand how this method works. This will make configuration and troubleshooting much easier.

Standard Procedure

First, let's talk about what happens in the standard setup (without any bypass). At a high level, the following process happens when the gateway boots up:

  1. All traffic on the ONT is protected with 802.1/X. So in order to talk to anything, the Router Gateway must first perform the authentication procedure. This process uses a unique certificate that is hardcoded on your Residential Gateway.
  2. Once the authentication completes, you'll be able to properly "talk" to the outside. However, all of your traffic will need to be tagged with VLAN ID 0 (a.k.a. VLAN Priority Tagging[1][2]) before the IP gateway will respond.
  3. Once traffic is tagged with VLAN ID 0, your Residential Gateway needs to request a public IPv4 address via DHCP. The MAC address in the DHCP request needs to match that of the MAC address that's assigned to your AT&T account[citation needed]. Other than that, there's nothing special about the DCHPv4 handshake.
  4. After the DHCP lease is issued, the WAN setup is complete. Your LAN traffic is then NAT'd and routed to the outside.

Bypass Procedure

To bypass the Residential Gateway using OPNsense, we can emulate the standard procedure entirely within OPNsense. However, there is a complication; tagging traffic with VLAN ID 0 is not supported through the standard interfaces.

netgraph

This is where netgraph comes in. netgraph allows you to break some rules and build the proper plumbing to make this work. In our case, it simply allows us to tag traffic with VLAN ID 0.

Smart/Managed Switch

Alternatively, you can setup a smart/managed switch between the ONT and OPNsense. Assign two ports to an untagged VLAN, choose an ID other than the default. I chose 100. You can use the rest of the switch ports as usual. I chose to use them as untagged VLAN 1.

D-Link:

image image image

TP-Link:

image image

Now traffic from the ONT should be stripped of its VLAN tag before being sent to OPNsense.

Hopefully, that gives you an idea of what we are trying to accomplish. See the comments and commands in bin/opnatt.sh for details about the netgraph setup.

Setup

Prerequisites

  • Certificates extracted from your Arris NVG589, NVG599, BGW210-700, or Pace 5268AC.
  • The MAC address tied to your certificates.
  • Local or console access to OPNsense.

There are some excellent resources on rooting and extracting certificates from various gateways:

Install

  1. Download/clone this repo somewhere you can edit it.

  2. Copy your certificates somewhere they can be accessed by the script, such as the wpa directory.

  3. Edit the configuration variables in bin/opnatt.conf to your setup.

# Interface Options

ONT_IF="xx0"
EAP_IDENTITY="XX:XX:XX:XX:XX:XX"
RG_ETHER="XX:XX:XX:XX:XX:XX"

# wpa_supplicant Options

ca_cert="/conf/opnatt/wpa/ca.pem"
client_cert="/conf/opnatt/wpa/client.pem"
private_key="/conf/opnatt/wpa/private.pem"
  1. Copy the repo to /conf/opnatt:
scp -r opnatt root@OPNsense:/conf/
  1. At this point testing the script would be a good idea.
ssh root@OPNsense
/conf/opnatt/bin/opnatt.sh
  1. To start opnatt at boot:
ln /conf/opnatt/bin/opnatt.sh /usr/local/etc/rc.syshook.d/start/94-opnatt

  1. Assign ngeth0 as your WAN interface.

  2. Once again, it's recommended to reboot OPNsense to test that WAN comes up.

  3. Apply your configurations, firewall rules, etc. in OPNsense. Do not add the underlying interface (xx0) to OPNsense.

IPv6 Setup

Big thanks to @pyrodex1980's post on DSLReports for notes on IPv6.

WAN Setup

  1. Go to Interfaces > WAN
  2. Enable IPv6 Configuration Type as DHCP6
  3. Scroll to DCHP6 Client Configuration
  4. Enable Request only an IPv6 prefix
  5. Enable DHCPv6 Prefix Delegation size as 60
  6. Enable Send IPv6 prefix hint
  7. Enable Do not wait for a RA
  8. Save

LAN Setup

  1. Go to Interfaces > LAN
  2. Change the IPv6 Configuration Type to Track Interface
  3. Under Track IPv6 Interface, assign IPv6 Interface to your WAN interface.
  4. Configure IPv6 Prefix ID to 0. You CAN use IPv6 Prefix id 0, as OPNSense does NOT assign a routeable IPv6 address to ngeth0
  5. Save

If you have additional LAN interfaces repeat these steps for each interface except be sure to provide an IPv6 Prefix ID that is not 0 and is unique among the interfaces you've configured so far.

DHCPv6 Server & RA

  1. Go to Services > DHCPv6 Server & RA
  2. Enable DHCPv6 server on interface LAN
  3. Configure a range of ::0001 to ::ffff:ffff:ffff:fffe
  4. Leave Prefix Delegation Range blank.
  5. Configure Prefix Delegation Size to 64
  6. Save
  7. Go to the Router Advertisements tab
  8. Configure Router mode as Stateless DHCP
  9. Save

If you have additional LAN interfaces repeat these steps for each interface.

That's it! Now your clients should be receiving public IPv6 addresses via DHCP6.

Troubleshooting

Logging

Output from opnatt.sh can be found in the OPNsense Web UI under System -> Log Files -> General.

wpa_supplicant

Connect to the wpa_supplicant daemon and view status:

wpa_cli status

tcpdump

Use tcpdump to watch authentication, VLAN and DHCP traffic.

tcpdump -ei $ONT_IF

Verify you are seeing 802.1Q (tagged as vlan0) traffic on your $ONT_IF interface and untagged traffic on ngeth0.

Verify the DHCP request is firing using the MAC address of your Residential Gateway.

If the VLAN0 traffic is being properly handled, next OPNsense will need to request an IP. ngeth0 needs to DHCP using the authorized MAC address. You should see an untagged DCHP request on ngeth0 carry over to the $ONT_IF interface tagged as VLAN0. Then you should get a DHCP response and you're in business.

If you don't see traffic being bridged between ngeth0 and $ONT_IF, then netgraph is not setup correctly.

Promiscuous Mode

opnatt.sh will put $ONT_IF in promiscuous mode via /sbin/ifconfig $ONT_IF promisc. It's unclear if this is required or not in supplicant mode.

Kernel Modules

Confirm the required kernel modules are loaded with kldstat. The following modules are required:

  • netgraph
  • ng_ether
  • ng_eiface
  • ng_vlan

You can edit which kernel modules are loaded by opnatt.sh in opnatt.conf.

netgraph

The netgraph system provides a uniform and modular system for the implementation of kernel objects which perform various networking functions. If you're unfamiliar with netgraph, this tutorial is a great introduction.

Your netgraph should look something like this:

netgraph

In this setup, the igb3 interface is my $ONT_IF.

You can generate your own graphviz via ngctl dot. Copy the output and paste it at webgraphviz.com or render locally with dot.

Try these commands to inspect whether netgraph is configured properly.

  1. Issue ngctl list to list netgraph nodes. Inspect opnatt.sh to verify the netgraph output matches the configuration in the script. It should look similar to this:
Name: igb3 Type: ether ID: 00000004 Num hooks: 1
Name: vlan0 Type: vlan ID: 0000000d Num hooks: 2
Name: ngeth0 Type: eiface ID: 00000010 Num hooks: 1
  1. Inspect the various nodes and hooks:
  • igb3:
Local hook Peer name Peer type Peer ID Peer hook
lower vlan0 vlan 0000000d downstream
  • vlan0:
Local hook Peer name Peer type Peer ID Peer hook
vlan0 ngeth0 eiface 00000010 ether
downstream igb3 ether 00000004 lower
  • ngeth0:
Local hook Peer name Peer type Peer ID Peer hook
ether vlan0 vlan 0000000d vlan0

Reset netgraph

opnatt.sh expects a clean netgraph before it can be ran. To reset a broken netgraph state, try this:

/usr/sbin/ngctl shutdown waneapfilter:
/usr/sbin/ngctl shutdown laneapfilter:
/usr/sbin/ngctl shutdown $ONT_IF:
/usr/sbin/ngctl shutdown $RG_IF:
/usr/sbin/ngctl shutdown o2m:
/usr/sbin/ngctl shutdown vlan0:
/usr/sbin/ngctl shutdown ngeth0:

This will remove netgraph nodes setup by other bypass modes e.g. netgraph bridge. Make sure to define $ON_IF and $RG_IF before running!

Other Methods

pfatt (netgraph w/ bridge)

https://github.com/MonkWho/pfatt

Linux

If you're looking how to do this on a Linux-based router, please refer to this method which utilizes ebtables and some kernel features. The method is well-documented there and I won't try to duplicate it. This method is generally more straight forward than doing this on BSD. However, please submit a PR for any additional notes for running on Linux routers.

VLAN Swap

There is a whole thread on this at DSLreports. The gist of this method is that you connect your ONT, RG and WAN to a switch. Create two VLANs. Assign the ONT and RG to VLAN1 and the WAN to VLAN2. Let the RG authenticate, then change the ONT VLAN to VLAN2. The WAN the DHCPs and your in business.

However, I don't think this works for everyone. I had to explicitly tag my WAN traffic to VLAN0 which wasn't supported on my switch.

FreeBSD

I haven't tried this with native FreeBSD, but I imagine the process is ultimately the same with netgraph. Feel free to submit a PR with notes on your experience.

U-verse TV

See U-VERSE_TV.md

Known Issues

  • ACME client cron job causes loss of WAN connectivity
  • Netflow may or may not work with ngeth0. Please submit feedback!
  • It's unclear wether $RG_ADDR is actually needed. Please submit feedback!
  • It's unclear wether promiscous mode is actually needed. Please submit feedback!
  • An option to log to /var/log/opnatt.log in addition to syslog would be preferred.
  • Occasional disconnects. Unclear why this happens, but copying the script to /usr/local/etc/rc.syshook.d/monitor will re-connect automatically.

References

Credits

This took a lot of testing and a lot of hours to figure out. If this helped you out, please buy us a coffee.

About

Use your own OPNsense router with AT&T Fiber.

Resources

Readme
Activity

Stars

3 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

No packages published

Languages

  • Shell 100.0%