Home Hub 4000 bridge drops internet when using the full 3Gbps pipe

[ninja0n3]

Hello,

I just spent the last 1h45 with a support rep and not any closer to finding a solution.

I'm curious to see if anyone else has experienced something similar.

Here is some context for reference: I have around 25+ yrs in IT/Infosec experience developing network infrastructures. I run a number of servers at home for both professional and personal reasons.

I currently have a 3Gbps Bell Fibe connection, this is running through the bridge port on the HH4k into an opnsense system using the 10G WAN port going into a 10G port on the firewall. The internet uplink is achieved using PPPoE (with Bell login info) on the 10Gb WAN port.
Up to here, everything works flawlessly:

My speed tests on the opnsense box are consistently, which matches the modem speed tests. This shows I am able to access the full 3G most of the time, clocking at at around 3.2Gbps for download and 2.8-3.2Gbps on upload.

For professional reasons along my line of work, I wanted to download a fairly large data set from one of our private AWS S3 servers (using s5cmd to speed up the process, which is able to max out a 40Gbps connection if available). The download is done on a file server with a 10Gbps line plugged into a 10Gbps switch, which is also connected to the opnsense firewall (thus providing internet access).

The first time I did it, the download ran for about 7-10 min, then all of a sudden the entire connection dropped - The internet went down through the PPPoE uplink/bridge.

The uplink was showing green on both the modem and opnsense (both different PPPoE lines). I was seeing around 400-425Mb/s (roughly the full 3Gbps in download) during the download process until the drop (I was able to access the modem through a separate ethernet port on my system).

Two minutes later, the internet returned by itself without any action. When I attempted the process again and every attempt after, the download went for around 4-5min consistently then same thing, connection drop. 2 Min down, then back online.

In my experience, this looks a lot like some kind of throttling or IPS/active defense blocking the connection after a certain sustained download threshold for size and time.

When I discussed this with 3 different reps, and after trying to explain the layout of the network, they kept insisting that I was having a hardware issue between the modem and the firewall. This was unlikely given I never lost connection to the upstream.

Eventually I managed to talk to someone from Level 2, which took another 30 min of explaining. After making me restart the modem 3-4 times, reboot the firewall and consistently encountering the problem 3-4 more times, he decided to contact his colleagues at SME Service, who were able to look at the packets and confirmed the connection was dropping.

The rep ultimately said that his SME Service colleague noticed the drops and mentioned the modem could be bad (noting that this is a replacement I received 3 days ago). They are shipping me a replacement.

After the call I attempted limiting the download bandwidth, to around 2.5Gbps on that download, but I encountered the same result.

I am not convinced another modem will make any difference; I feel like this is some kind of distribution center firewall/ips rule that blocks the kind of transfer that I need to perform at this point - This test transfer was around 500Gb, the full set is around 60Tb.

I'm not too happy with the fact that this is not working, especially since the rep kept repeating that there shouldn't be any limitations to bandwidth, duration or amount of data being transferred. This assumes I should be able to leverage the full pipe when available without restrictions.

I frequently transfer files around 10-15Gb without any problems at around 60-90Mbps. This issue is pretty strange.

I know this is a very particular use case, but I was wondering if anyone else was experiencing the same kind of issues that I am seeing with sustained full pipe transfers. Or if anyone could provide any insight on something config related that I could have missed that could be causing this problem.

Any thoughts, ideas appreciated.

Thanks!

[ZaneP]

Hi @ninja0n3 

Your issue likely goes well beyond the scope and training of Bell residential tech support personnel who deal with "average customer" problems.

The issue of Bell throttling bandwidth or otherwise interfering in data transfers has been discussed on many forums and there's been zero evidence offered to support the notion. Perhaps there is a back-end rule in play which disrupts full pipe transfers at the scale you need, but isn't applied to enterprise customers. Even as I type this, that seems unlikely!

I have nothing to bring to the table on your use case, but here's some suggestions:

• post your scenario and issue on the Bell Direct forum at DSL Reports. It's a one-on-one private conversation with very experienced Bell tech support people who monitor the forum. https://www.dslreports.com/forum/sympatdirect
• also post on the general Bell forum at DSL Reports. There's a discussion on PFsense going on which may be relevant, (or not 🙂); you can also do a search through the topics for OPNsense, etc. https://www.dslreports.com/forum/r33439081-Internet-pfSense-CPU-Sizing
• do a similar post on one of the OPNsense forums. There may users with Bell or other ISPs who have had similar problems and can offer some ideas
• be persistent in escalating the issue with Bell; the Community mods here might be able to facilitate your connecting to the appropriate team

Good luck and keep us posted!

[ninja0n3]

Hello @ZaneP 

Your issue likely goes well beyond the scope and training of Bell residential tech support personnel who deal with "average customer" problems.

I agree, the problem is technical in nature and I understand the limitations of residential support. I still feel that given the feature is available to residential customers, they should at least be able to move a customer up the support chain to get to someone who has the technical capability to address these issues.

The issue of Bell throttling bandwidth or otherwise interfering in data transfers has been discussed on many forums and there's been zero evidence offered to support the notion.

Fair enough, throttling is probably not the best choice of word for the problem, since I am operating at full bandwidth. Some kind of IPS or usage-based blocking rule is what I'm referring to.

Perhaps there is a back-end rule in play which disrupts full pipe transfers at the scale you need, but isn't applied to enterprise customers.

There lies the problem, no one is able to tell me; changing the modem feels like a catchall attempt at resolving the issue. This was suggested by the SME Service that the Level 2 support rep talked to.

My expectation, and also based on what the rep said, is that there should be no restrictions on the connection, which includes no blocking rules when I'm leveraging the full pipe. I'm using a residential line, but that shouldn't be a factor in being able to use the connection.

If the modem is indeed at fault, then I'll be happy to just move on and close the issue. I just feel that, based on the consistency of the connection drop out under the same conditions, a modem issue is unlikely. I also doubt any blocking rules would be hard coded into the HH4k modem.

• post your scenario and issue on the Bell Direct forum at DSL Reports. It's a one-on-one private conversation with very experienced Bell tech support people who monitor the forum. https://www.dslreports.com/forum/sympatdirect
• do a similar post on one of the OPNsense forums. There may users with Bell or other ISPs who have had similar problems and can offer some ideas
• be persistent in escalating the issue with Bell; the Community mods here might be able to facilitate your connecting to the appropriate team

Thanks for sharing that. I will reach out to those forums, I didn't know about the bell direct forum, great resource.
I didn't expect a response from bell tech, but rather trying to see if anyone had ever experienced anything along those lines.

[ZaneP]

Hi @ninja0n3 

When I read your original post I also felt the modem was not the issue and it's a waste of time getting replacements. (It can't be entirely ruled out: the HH4K firmware is dumbed down - for example it has locked out users from configuring its firewall ; it can't be bypassed unlike the HH3K)). 

Bell needs to be transparent on any significant technical matters that differentiate residential and enterprise customers, other than the obvious ones (like managed services).  

Hopefully you will have a productive conversation with a higher-level Bell tech soon. Meantime you might get a valuable response from the Bell direct forum.

[ninja0n3]

Update on this thread.

I received the replacement unit and as expected the replacement modem did not work...

I am fairly certain Bell is blocking the internet once it reaches a certain threshold, as I am able to ping the PPPoE gateway when the internet drops during the download process: I have no access to the internet but I am able to ping the external PPPoE gateway IP that is assigned.

Both screenshots below were taken while the internet was down.

I'm currently on a call with Bell support and for some reason they're consistently reluctant about putting me through to someone who actually understands how their equipment, ethernet or PPPoE works.

One of the most ridiculous suggestions I've received today is that the website is blocking my download... This is what the reps are getting told to tell me. I highly doubt an AWS S3 download would block my internet altogether and let me ping only the PPPoE gateway IP.

So while Bell may not be throttling me, they certainly don't let me use the full speed I'm paying for. They are somehow blocking my internet connection at the gateway/distribution center level (which I am still able to contact through ping when the internet drops).

So far no one has been able to provide me a straight answer about this. I'll keep digging and working with them in the hopes of finding a solution.

What's concerning is that Bell offers a 3Gbps service, which I am able to confirm works (speed tests clocking at 3.2Gbps up and down), the equipment they offer is inadequate to leverage that speed fully, since it either doesn't have fast enough ports or eats away at the bandwidth.
If you try to use the bridge feature, which is definitely supported by the equipment, to leverage the speed, then there are seemingly blocks in place that prevent the actual sustained use of the line at 3Gbps.

Support is clueless as to how to handle any of this.

[ninja0n3]

Update on this thread.

I've spent over 6h on this case altogether working with support.

The level 3 reps were great and tried their best, they had a better understanding of what I was discussing and were able to attempt to get some insight by discussing this with their more technical colleagues.

Unfortunately it seems no-on at bell is able to provide any information beyond what happens at the modem level and seemingly don't have access to any actual networking infrastructure, including their gateway centers. I found this to be pretty surprising, and taking into account they were legitimately trying to help, the could not even tell me who I could get in touch with, they simply didn't know.

I appreciate their assistance with this, they did great in trying to help.

As far as Bell goes, I am happy that they're increasing they're speeds and all, but they lack any kind of technical staff capable of supporting anything beyond basic needs. I feel like it should be possible to reach someone who could peak into the gateways and have a look at the networking stuff when it gets really technical... At the end of the day, I'm not really able to use what I'm paying for in terms of service; taking full advantage of my connection is constantly blocked.

My guess is that there's some DDoS protection IPS rule that is triggered by a false alarm due to the number of packets being transferred and ultimately temporarily places a restriction on the node for a period of time.

It's a very disappointing, more so that I still don't have any answers, and I have no path forward to get this resolved.

[ninja0n3]

Update on this thread, after over a month of messing around and nearly 25 hours of going back and forth with all kinds of people working for Bell, they essentially told me to use it as it is or go somewhere else before hanging up on me.

That last manager was probably one of the rudest people I've had the misfortune of talking with.

My feeling is that he gave me a call back to get rid of me and forcibly close my case.

I'm our of recourse at this point, Bell made it clear they are not willing to investigate any further and that I either take their service as is (in spite of the fact that they keep on advertising there are no restrictions) or find another company. They suggested Videotron! 

Unfortunately Bell has the monopoly on high speed fiber internet, so not really a choice.

In any case, a full writeup here discussing the entire case, the supporting proof and logs that show the issue: https://www.dslreports.com/forum/r33444694-Internet-Home-Hub-4000-bridge-drops-internet-at-3Gbps

Thanks for reading.

@ZaneP Thanks for trying to help, greatly appreciated.

[cmon]

whenever  I'm downloading a large amount of file, bell internet just disconnects for no reason. I'm trying to restore some back ups, but have made no progress.

[BellPatricia]

Hey there @cmon. Thank you for your post, and welcome to the Bell Community.

Just a few questions so the Community can better assist you.

• Which modem are you using?
• What is the size of the file you are downloading, and can you share where you are downloading the file from? 
• Have you tried downloading the files on a different computer? 
• Are you running a VPN or any third party equipment such as a router? 
• Are you using a wired Ethernet connection or wireless?

Keep us posted.

- Patricia 

[Anton]

Can someone please explain to me, why my connection drops for 10 minutes if i download at full speed

your phone support is uselles, all they can do is change the modem 

8GBps connection

iperf3 runs fine for a long time on 9.48Gb/sec without any problems

while downloading 1TB- speed is ~ 600-650MB/sec, after 20 minutes network connection shuts down and restarts after 10min (modem is not shutting down or restarts, i have access to modem)

Please look at the logs, last disconnect (8/13/2023  7:50PM)

[dks]

It’s is a known issue for the 8Gb profile which has been reported in the DSL Reports Bell forum. No idea if Bell is looking at it. 

[BellPatricia]

Good afternoon @Anton.

Thank you for reaching out to the Bell Community.

Can you please share a bit more information with the Community, as per our questions in the thread above?

Looking forward to your reply,

- Patricia

[ninja0n3]

I have a much lengthier thread on DSL Forums on the subject about this, we're at about 10 pages worth of conversation: https://www.dslreports.com/forum/r33444694-Internet-Home-Hub-4000-bridge-drops-internet-at-3Gbps

The takeaway is that Bell is unwilling to acknowledge the issue, let alone try to fix it. I don't think they'll do anything about it unless this gets enough exposure and/or ends up in the news.

My issue has been going for over a year now and as more customers have gained access to 3G/8G, this is becoming more evident.
I spent over 60 hours working with their support.

In any case, good luck.

[dks]

Yes, I have been following that thread on DSLR. It reminds me of the conversation which was ongoing with dial-up and DSL internet services. Speeds are always "up to..." That caveat is still in Bell's advertising for fibre services, although it is not very prominent.  

[ZaneP]

As @ninja0n3 wrote, the issue is well documented on the DSLR Bell forum. 

Distilling 10 pages of discussion is difficult, but a credible and knowledgeable contributor on that forum wrote this two days ago:

"the only known mitigations at this time are to limit download speed or use multiple pppoe sessions either in series or parallel to minimize the impact of potentIal failures". 

 To open multiple ppope sessions you'll obviously need appropriate multi-WAN hardware connected to Bell's gateway modem. @Anton and @cmon, you may already have this (?).