News & Events

Introduction and issues with remote access under Carrier Grade NAT (CGN)

With the proliferation of smartphones, mobile apps and connected devices comes the expectation that customers will be able to access their devices at home or work remotely - particularly alarm panels, CCTV NVRs - and in the future - smart home automation controllers and sensors.

However, there are many network and device issues that can break the end-to-end connectivity from a remote client back to their residence and some trade-offs that you and your customers should be aware of with the various solutions to these issues.

IT Manager Mark Roemermann

IT Manager Mark Roemermann

Introduction

As always with so much of IT, it is a matter of diagnosing the root cause and recommending a solution with the awareness that depending on the customer's circumstances, cost or complexity of implementation you might have to compromise on the “least worst” combination of expense and functionality to solve the problem.

Over the next couple of months I would like to outline some of the issues you might have onsite with customer networks when configuring your WiFi or IP CCTV equipment for remote access.   I will aim to keep these articles as user friendly as I can for a non-network engineering audience (and include lots of links to external resources for your own followup education), but as you will see there is no hiding a certain level of complexity and inter-related factors. 

If you have any feedback on this series of articles, suggested topics you would like covered or if you would like to see NAS run another introduction to IP networking course, please contact me at mark@nasaustralia.com.au.

Today I’ll briefly outline the issue of Opens external link in new windowCarrier Grade NAT (CGN), the reasons it exists and a few of the possible solutions to it.

Carrier Grade NAT (CGN)

Opens external link in new windowCarrier Grade NAT (CGN) also called Large Scale NAT (LSN) can break Opens external link in new windowend-to-end connectivity between a remote client trying to connect inbound to a residential IP address.  The ISP effectively places a NAT router (a much larger version of what you have at home) Opens external link in new windowbetween their customers and the Internet.  Internally on the customer facing side, you receive a private IP address (eg: 10.2.103.401; an address from the Opens external link in new windowRFC1918 address space) while on the Internet facing side the ISP uses one of their publicly routable IP addresses.  As traffic is initiated outbound from the customer to the Internet, the CGN device (also Opens external link in new windowcalled a middlebox by network engineers) translates your private IP address assigned to your router to one of the ISP’s publicly (also known as ‘globally’) routable IP addresses.  The CGN device tracks this translation internally so that any reply traffic from the website you’re connecting to is sent back to the correct private IP internally.  In just the same way that we have used NAT routers at home to hide many devices with private IPs (multiple laptops/PCs/smartphones/printers/TVs) behind a single public address, an ISP uses CGN to hide many of their customers behind a single - or a small range of - public IP addresses.

What does this mean for your customers mobile app trying to connect back to their NVR at home?

Well it means that in a Opens external link in new windowlot of cases a connection initiated from outside the ISP has no way to connect back to the intended home destination as the ISP’s CGN device does not know how to map a public address back to a private address without a corresponding initial outbound connection.  For all intents and purposes in layman terms, CGN only allows outbound connections and replies from customer networks.  

So why on earth have ISPs started to do this?

Opens external link in new windowAs mentioned by APNIC (the organisation responsible for allocating ISPs with IP addresses in the Asia Pacific region) this is a recent issue that has cropped up as ISPs use CGN to deal with the Opens external link in new windowproblem of IPv4 address exhaustion.  You can read up on it by following the links, but effectively publicly routable IP addresses are becoming a scarce resource.   ISP’s are now Opens external link in new windowbuying and Opens external link in new windowtrading IPv4 addresses on open markets from regions with reserves of public IP addresses (eg: Africa) Opens external link in new windowinto regions that have very little IPv4 address space left (eg: Asia Pacific and North America).

CGN is Opens external link in new windowone of the tools in their toolbox to prolong the availability of IPv4 addresses on their networks, it’s just a tool that has unfortunate side-effects for residential users desiring remote access from the outside.

How can you determine if your ISP is using CGN?

From what I’ve heard from customers, this has become more common on some providers cable and NBN plans.  It is also almost universal on personal/residential 3G/4G mobile internet plans that are not business grade (Opens external link in new windowsee this attached Telstra presentation from APNIC conference).  I have not heard as many instances of it occurring on ADSL plans yet, although as the IPv4 crunch starts to ratchet up I can see it being used more and more.

Determining if you are behind CGN is pretty easy.  You are looking for a mismatch between the IP address on the WAN port of your customers router and the ‘publicly visible IP address’ as determined by a remote third party website.  

As an example, I have a mobile with Telstra.  

  1. I open an Opens external link in new windowapp on my iPhone called “istat” to show me that the IP address assigned to my phone from the local cell tower is 10.201.53.136.  I know this is an address from the Opens external link in new windowRFC1918 private address range of 10.0.0.0/8 so this already gives me a hint.  
  2. However, if I open up safari in iOS, and visit the website http://mxtoolbox.com/whatismyip/ it will show me that my publicly visible and routable IP address is actually 182.239.220.85.  
  3. This indicates that Telstra is operating a CGN device to translate from my phone at 10.201.53.136 to 182.239.220.85 out to the internet.

This is rarely an issue on a mobile phone as the majority of that traffic is initiated outbound from the phone to the internet and the CGN can track the replies to the requests and send them back to the originator.  However it can be problematic on fixed residential modems that might want to host a web server or offer remote access.

A secondary issue that is caused by CGN is ‘Opens external link in new windowdouble NAT’.  Double NAT is where there is two (or more) layers of NAT between the endpoints of a connection.  In the residential case, your home router performs NAT from your local network to it’s WAN address, and the ISP’s CGN device performs NAT between their private customer network and the public Internet. 

This is particularly problematic for VOIP calls, and other traffic that relies on UDP packets that might be initiated inbound.  You often see the symptoms of this as one-way audio in VOIP calls, and other strange connectivity issues as the CGN device needs to do some guessing about how to handle certain types of traffic using protocols that weren’t designed with double NAT (or any NAT really) in mind.  The protocol designers in a lot of cases assumed end-to-end connectivity would be available. 

So what can you do about it?

There aren’t many solutions to this problem that are directly in your control.  Some require a network engineer, some would require developers to modify their applications, or provide a third party service.  

  1. By far the least complicated solution at the moment is to ask that the customer requests that they be assigned, or upgrades to an internet plan that provides, a “publicly routable IP address”.  This could be an extra monthly fee for some ISPs, or it might require an upgrade to a business grade plan to receive a non-private IP address. From there you can do the usual port forwarding or VPN to access their NVR remotely (a future article).

  2. Another possible solution is to buy devices that connect to a cloud service on the internet and use that service to provide the endpoint for connection, or have a connection brokered between an app and a device.  An example of this is the remote support applications a lot of you use like ‘Team Viewer’ where Team Viewer’s servers act as a connection broker between a residential endpoint and a remote client both making outbound connections to Team Viewers servers.

    The catch with this solution is that you are reliant on the uptime and security of a third party.  It’s also not a particularly scalable solution as each smart device you buy would need to come with its own connection brokering cloud service that might get a little complicated to manage while opening up multiple connections from third parties on the internet into your private networks.

  3. An alternative solution for networking experts is to create a VPN tunnel from your home router (if it is capable of this) to a cloud hosted server on the internet that you’ve setup.  This remote server then becomes your publicly routable endpoint to connect remotely to.  This is obviously a little complicated to setup if you’re not an expert however, and would expose you to other IT support issues from the client if and when there were issues with the cloud hosted tunnel endpoint.

  4. Developers of applications have other options.  They can establish a connection brokering service as described above, or they can implement a variety of “Opens external link in new windowNAT traversal” techniques.  This is likely to become more common in applications as ISPs make greater use of CGN however, this requires application and protocol changes which aside from nagging your vendor, is not necessarily easy for you to accomplish. 

In theory over the next 10 years, IPv6 will gradually replace IPv4 and remove the scarcity of IP addresses as an issue and restore the end-to-end connectivity model, although there are battles being waged between various parties on the inclusion of NAT or network prefix translation (NPT) functionality in IPv6 - this is a topic for an advanced article in the future.

So what next?

Once you have a publicly routable IP address, you now have an easily reachable endpoint to connect your mobile app to.  From there you will still need to work out how to connect through the customers home router to their NVR on their internal network - a topic for a future article!