Category: Service

Connectivity issues from last week

The issues from last week resulting in some users experiencing intermittent web browsing drops, has been resolved.

At the start of July, RM installed a different version of Squid (software that runs over the proxy servers). Following the work, initial testing reported that the change was successful and the network was stable. However, when schools returned in September, the extra volume and types of traffic caused the load balancers to intermittently terminate internet sessions. Users could however refresh their browser and the service would then resume, but obviously this is far from ideal.

On Friday, RM subsequently rolled back from this change and all appears stable once again. We are continuing to monitor but please get in touch with us if you wish to discuss the matter further.

Thanks,

Kevin Crawley

Internet sessions dropping out

Good morning,

We have received several reports of internet sessions dropping out. RM are actively dealing with this.

Thanks,

Kevin Crawley

Firewall Upgrade

Following on from last night’s firewall upgrade, there are unfortunately some issues. There are some schools who have reported issues with their VOIP/SIP deployments and some Google services are also not working as expected. RM senior engineers are investigating.

Thanks,
Kevin Crawley

Speed issues

We believe this has now been resolved. It looks like there were two simultaneous issues;

  • RM’s transit to Google was being maxed out. RM will be implementing new routers with 100G links to deal with the increased traffic volumes. RM is hoping for this will be completed by the end of the month.
  • There were also six minor DDoS attacks to the proxy farm IPs at the same time of high utilisation of Google traffic. RM is working with a third party company to mitigate these attacks. As they were minor attacks, it didn’t trigger the automatic mitigation system. RM will be working closely with their third party to investigate what went wrong and to make sure this doesn’t happen again.

If there are any further queries, please contact the HfL Broadband Team.

Slow internet

Good morning.

We have received multiple reports of slow internet this morning. RM are investigating. Apologies for the disruption.

Thanks,
Kevin Crawley

Possible Google update and reports of slow internet

Yesterday afternoon (18th October) we received some reports of slow internet.

RM suspect that there was a Google update released which resulted in higher than usual volumes of traffic – and this in turn would cause a slow internet experience. Typically, the RM transit (for Google) utilises at a maximum of 8Gbps, but yesterday it went over 10Gbps.

In fact there were three instances yesterday where the transit bandwidth utilisation exceeded the capacity, timescales for which are updated below.

8.48-9.08
11.25-11.40
13.55-14.03

RM is closely monitoring the situation.