Home Hub 4000 bridge drops internet when using the full 3Gbps pipe

[ninja0n3]

Hello,

I just spent the last 1h45 with a support rep and not any closer to finding a solution.

I'm curious to see if anyone else has experienced something similar.

Here is some context for reference: I have around 25+ yrs in IT/Infosec experience developing network infrastructures. I run a number of servers at home for both professional and personal reasons.

I currently have a 3Gbps Bell Fibe connection, this is running through the bridge port on the HH4k into an opnsense system using the 10G WAN port going into a 10G port on the firewall. The internet uplink is achieved using PPPoE (with Bell login info) on the 10Gb WAN port.
Up to here, everything works flawlessly:

My speed tests on the opnsense box are consistently, which matches the modem speed tests. This shows I am able to access the full 3G most of the time, clocking at at around 3.2Gbps for download and 2.8-3.2Gbps on upload.

For professional reasons along my line of work, I wanted to download a fairly large data set from one of our private AWS S3 servers (using s5cmd to speed up the process, which is able to max out a 40Gbps connection if available). The download is done on a file server with a 10Gbps line plugged into a 10Gbps switch, which is also connected to the opnsense firewall (thus providing internet access).

The first time I did it, the download ran for about 7-10 min, then all of a sudden the entire connection dropped - The internet went down through the PPPoE uplink/bridge.

The uplink was showing green on both the modem and opnsense (both different PPPoE lines). I was seeing around 400-425Mb/s (roughly the full 3Gbps in download) during the download process until the drop (I was able to access the modem through a separate ethernet port on my system).

Two minutes later, the internet returned by itself without any action. When I attempted the process again and every attempt after, the download went for around 4-5min consistently then same thing, connection drop. 2 Min down, then back online.

In my experience, this looks a lot like some kind of throttling or IPS/active defense blocking the connection after a certain sustained download threshold for size and time.

When I discussed this with 3 different reps, and after trying to explain the layout of the network, they kept insisting that I was having a hardware issue between the modem and the firewall. This was unlikely given I never lost connection to the upstream.

Eventually I managed to talk to someone from Level 2, which took another 30 min of explaining. After making me restart the modem 3-4 times, reboot the firewall and consistently encountering the problem 3-4 more times, he decided to contact his colleagues at SME Service, who were able to look at the packets and confirmed the connection was dropping.

The rep ultimately said that his SME Service colleague noticed the drops and mentioned the modem could be bad (noting that this is a replacement I received 3 days ago). They are shipping me a replacement.

After the call I attempted limiting the download bandwidth, to around 2.5Gbps on that download, but I encountered the same result.

I am not convinced another modem will make any difference; I feel like this is some kind of distribution center firewall/ips rule that blocks the kind of transfer that I need to perform at this point - This test transfer was around 500Gb, the full set is around 60Tb.

I'm not too happy with the fact that this is not working, especially since the rep kept repeating that there shouldn't be any limitations to bandwidth, duration or amount of data being transferred. This assumes I should be able to leverage the full pipe when available without restrictions.

I frequently transfer files around 10-15Gb without any problems at around 60-90Mbps. This issue is pretty strange.

I know this is a very particular use case, but I was wondering if anyone else was experiencing the same kind of issues that I am seeing with sustained full pipe transfers. Or if anyone could provide any insight on something config related that I could have missed that could be causing this problem.

Any thoughts, ideas appreciated.

Thanks!

[Vanadiel]

I would run a winmtr to 1.1.1.1, start the download and observe the winmtr output window.

If it truly drops the connection, you should see it reflected in the output window as packet loss.

I have new world, and never had any issues downloading or patching it. the patches often take some extra time because it patches the +80 Gb game client with each update...

[Junrhod]

I had the issue with PTR game client, downloaded from scratch ! I'll try to run winmtr while redl it !

[Explorer]

I'm now experiencing this very same issue, and I think it is related to the speed somehow, so here is my situation: I got a new hardware with apparently a fast WiFi 6GH, it is hooked on the Bell Hub and working well. As I started my games required updates, not very huge some 48G which I did before on the old systems and on XBox with no problems, the update download goes for 10 minutes but then stops and the machine loses access to the Internet. Aside from a brief interruption for them, other machines in my house are still connected, have access to the Internet and working.

While I'm in this situation, I switch my WiFi to tether with my Cell and the download continues, I switch back and it stops again. Now comes the funny part, I have an WiFi access point, I turned on and swicthed my WiFi on the new computer to connect to it (which is eventually connected to the same Bell Hub) and the download continues, and of course the new computer resume internet access. This access Point is 2.4GH which is slower so I think that explains what is going on, It is in the Bell Hub that it is marking that particular WiFi connection from my new computer and not allowing it further access.

For now I'm using the Access Point in the middle and I have no idea how I can reset the Bell Modem/Hub to get back to normal operations for that new computer, which is done with the download by the way.

[Vanadiel]

This is different than the original posting, which said the Bell HH is loosing connection to the internet.

In your case, it's just the WiFi connected machine while the others stay connected. It could be due to a large amount of packets being dropped over WiFi, or an incompatibility issue between the HH and your WiFi card. You would have to perform some more troubleshooting on the WiFi side to get to the bottom of it.

[Explorer]

I admit you are right, at least you pushed me to troubleshoot further and I think I have a solution, my wireless network adapter supports up IEEE 802.11a/n/ac/ax and was picked by default, I tried with IEEE 802.11a/n/ac (no ax at the end) and it worked. Of course, it doesn't explain that it worked downloading about a quarter of my 48GB then stopped, but there might be something defective here.

Thanks for the tip.

[bimmer]

Well this keeps happening to me over and over every few days.. Very annoying. Just wish there was another company offering these speeds.

[dks]

There are a number of threads which address this issue on DSL Reports. There is no official Bell channel there, but the users are very knowledgeable. https://www.dslreports.com/forum/sympat 

[bimmer]

I'm curious, would you mind trying to download Warzone off Battlenet and see if you can get the whole thing at max speed without dropping connection?

[xenago]

Saw your post on the dslreports forum, I replied there but might as well here too.

This matches my experience on 3/3gpbs in Ottawa area. As mentioned, the problem usually results in 6+ minutes of downtime - I've seen cases where it approaches 10 minutes! This never happened when I was using the HH3k on GPON with ~gigabit speed.

Ever since I 'upgraded' from 1.5/940 on hh3k to 3000/3000 on gigahub, this has been a constant issue. For a while I thought it was a problem with my routing hardware, as I swapped from a Ubiquiti ER-12 to x86 OpenWRT around the same time and the problem only occurred when using OpenWRT. However, that was a red herring - the EdgeRouter 12 is limited to 1GbE, which appears to be too slow to hit the issue.

Here is an excerpt from the router system log when it occurs. I use `watchcat` to ping Cloudflare 1.1.1.1 as a status check, which ends up serving as a good example for how long the outages last when the problem hits:

Sometimes the disconnect even occurs repeatedly just minutes apart:

The assigned dynamic IP address doesn't change/renew when the connection comes up, and the gigahub appears to remain in some kind of working state (can even run the built-in speedtest while the external router's internet access is down).

There is no point in paying more if I can't use the increased bandwidth without torpedoing my internet connection.

They really need to sort this problem out as it negates the only benefit of the higher-priced plans. Obviously, if anyone from Bell is reading this, I would be happy to send logs.

[Vanadiel]

See below. There was some other network traffic at the time I was downloading this so it's slightly slower than the gigabit connectivity allows on my network, but it was consistent and there were never any network drops.

Both the movies we were streaming were happily streaming while this +86 GB download was going on.

I don't have a HH4000 to test the theory it drops connectivity at high speed downloads.

[bimmer]

Thanks for trying, in my experience it happens when downloading closer to 200mbps or more.

[Vanadiel]

I cannot reach those speeds on my plan, so hopefully someone else can test it out for you.

[dks]

If you are following the issue on DSL Reports, there are also threads on reddit and discord. I don't have links, unfortunately. It appears to be a known issue with no fix yet. 

[MommySciku]

I don't know why this keeps happening, but downloading anything bigger than 20GB shuts my net off, same thing happens if I reach a speed of 125MBps. The fact that there's no official fix, or not a fix at all is infuriating.

As soon as it reached 20GB on Forza, goodbye internet, TV and Phone. I swear I'm just paying for no service.

[Vanadiel]

I switched things up a bit and now use a Mikrotik RB5009 with the GPON module directly plugged into it.

Perfectly stable at 1.5/1. Just an update for those who were following the thread.