Home ยป How to optimise your network/webste/application during these high traffic pandemic times

How to optimise your network/webste/application during these high traffic pandemic times

Many ISPs dont have much access to international bandwidths too, wifi and mobile shared architecture
With the pandemic, most people are staying at home and using the internet more. During these times, backbones get taxed a lot more and exposes a lot of ISP’s bad infrastructure and setup. There are many ways to optimise traffic but sadly most ISPs in the world have shot themselves in the foot when giving internet users a good experience. Many ISPs and some countries also do not have control over their international or exchange bandwidth with other ISPs and countries.

Most ISPs in the world still give very low uploads and focus on downloads. This is very counter productive since you want more home p2p happening without having to touch your backbone, and having the usual high bandwidth links from datacenters used well. To achieve this CDNs and caches near the user are very important, and had uploads not been poor more products could’ve targeted the home user to having their own services such as their own file and media servers which could’ve done something like download a series they would want to watch overnight when people are sleeping or not using the internet much.

As a home user, the best you can do is to learn to host your own media and files while utilising cloud services. You can also set up a cache for your network like this:

In this diagram, having a cache for multiple users is very helpful both for homes with multipler users, ISPs and even offices as well. Having a cache not only for web but other content such as game distribution platforms like steam and OS updates which have also evolved to use p2p will benefit greatly from this in reducing bandwidth contention on the main links. Currently NGINX remains the best to create a proxy with due to its compatibility with many different applications and protocols.

QoS is another important factor as well that should be considered on many levels at home and for the ISP. ISPs should not throttle connections but rather only prioritise and only throttle low priority connections when links are full. For home users they should apply QoS based on their usage. It is important to understand that applications require specific things to function properly. For example Videos can have their packets processed late so can take any path as long as their bitrate is matched. A video of 2Mb/s only requires 2.2Mb/s of bandwidth (accounting for a bit of overhead) which you can restrict via bandwidth limit per video stream but will cause buffering when part of the video is skipped. Gaming and VOIP require little but consistent amounts of bandwidth and lowest latency for example, so should be given priority and the path taken should be prioritised to be the fastest path (rather than highest bandwidth path).

A typical home internet could be downloading p2p or torrents while watching a video with another user browsing the internet while being in a video call at the same time another user could be gaming and while this is going on, their systems may be updating at the same time and their files syncing with the cloud. Its important to understand that all applications use both upload and download at the same time but differently, and different protocols work differently with some necessary protocols for the internet to work such as DNS and NTP as the basics. If you can plot your QoS stack you can optimise your internet well, and prioritising the basics like DNS queries and using a fast DNS server can speed up many applications.

A well employed QoS can ensure applications get the bandwidth their need and for time sensitive applications, have their packets processed ahead of the queue. Low prioirty services like ping would seem slow which some newbie skill benches like dslreports measure bufferbloat via ping. High bufferbloat may not necessarily be bad if you employ QoS as its easy to cheat dslreports’s bench for an A+ bufferbloat rating, but the bufferbloat of the application you are prioritising is whats important, understanding that some applications can have bufferbloat (like file downloads, web browsing, video streams) but not latency sensitive applications like gaming, VOIP, NTP.

VPN services that offer compression barely assist in lowering bandwidth use noticeably. Images and video content are already compressed while text can be compressed loads more but web browsing which tends to contain a lot of code in text form, if the webserver isnt already compression it with the best available (brotli, gzip) then please email the admin of the webserver and mention how much of a noob he is in his job that he shouldnt be doing it. Compression is the most basic option i enable in every single website i handle, and if you are an admin for any website, you should too for all content as serving a page 4x less in size for the same content saves 4x more bandwidth. Think of those mobile uses who live on expensive mobile plans where even 1GB of data is very pricey too. smaller page sizes do load faster too!

As a content provider or server admin, use compression to the fullest. Many CPUs have compression accelerated by hardware so you have no reason not to use it. You also have no excuse to make unnecessarily large pages that are 10MB in size, as a focus i always aim for below 1MB in page sizes and if you absolutely must, keep it below 5MB for content pages that must have a lot of high quality images or use a lazy loading gallery that loads the content on demand instead. The key to reducing size by not only compressing but also providing scaled versions of content and with higher compression (avoid using quality or lossless except for the content itself delivered via a different page or way) and to avoid placing these content on pages that have to load other things, rather give large content their own pages with no other elements that need to be loaded to reduce contention. Minify your code as well as to reduce its footprint. The less you transfer the faster your page can load even during times when cloudflare is overloaded.

Use a CDN where possible. If you are a content provider like a website owner, use a CDN for it. Theres no reason not to and even OS use various servers to provide updates, no reason for you not to while getting the benefit of faster loading times while using less links to deliver your content. Many ISPs throttle international bandwidth that if you can avoid using it your content will get through even during times of contention. Make sure to host your content in the same country as your audience as well.

Make sure to test your web page to ensure it loads fast and well. gtmetrix, webpagetest, googlepage insights and pingdom can show you various details of your site that you can improve on it, so dont neglect the testing and do what you can to reduce wastage of both unnecessary code and content.

Leave a Reply

Your email address will not be published. Required fields are marked *

five × 4 =