Ubuntu 18.04 Chronicles: Applying firewall rules on startup, Pre-network

Another installment in this series. If you are used to dropping scripts in /etc/network/if-pre-up.d/ and seeing them get executed just before the network subsystem is up that’s another thing that doesn’t work in ubuntu Bionic, and you get no feedback that it doesnt.

Don’t panic though, because here I show you how to accomplish partially similar results. I say partially because my proposal here executes only on system boot, but that should suffice, because the firewall rules don’t disappear and need to be reapplied due to network status changes, and you will probably have other mechanisms in pace to deal with events that are related to network anyway.

The straightforward answer is that you now need to create a systemd service, which executes the script you would normally place in /etc/network/if-pre-up.d/. As a digression, despite the fact that systemd renders much of my previously acquired know-how useless, I actually like the logic of it’s design and so I hope that the new knowledge I’m acquiring and sharing here will be useful for long into the future.

Here’s my firewall rules script that I shamelessly adapted from Ars Technica: (the rules continue to be relevant to ubuntu Bionic, but the methods aren’t).

root@sol:/home/nucc1# cat /etc/network/iptables 
#!/bin/sh
echo "Loading Firewall Rules..."

WAN="enp3s0"
LAN="enp4s0"

logger "ROUTER: WAN: $WAN, LAN: $LAN"

logger "setting up base iptables rules"

/sbin/iptables-restore <<-EOF
*nat
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]

-A POSTROUTING -o $WAN -j MASQUERADE

COMMIT

*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]

#---------
#SERVICE RULES
#--------------------

#global accept rules
-A INPUT -s 127.0.0.0/8 -d 127.0.0.0/8 -i lo -j ACCEPT
-A INPUT -p icmp -j ACCEPT
-A INPUT -m state --state ESTABLISHED -j ACCEPT

#enable traceroute rejections to be sent.

You need to create a systemd unit file in /etc/systemd/system/ with the contents below (give it any name of your choice dot service, I call mine ‘router-rules.service‘):

[Unit]
Description = Apply base firewall rules for router functionality

[Service]
Type=oneshot
ExecStart=/etc/network/iptables

[Install]
WantedBy=network-pre.target

It’s pretty easy to understand I think. Type=oneshot means, execute the script and don’t try to daemonise it or something. WantedBy=network-pre.target is the systemd way of saying to execute something just before the network is configured, which is Pre-Network.

Once this systemd service has been created, you need to enable it (otherwise, it won’t run at startup).

sudo systemctl enable router-rules.service

Et voilà! Next time your system reboots, the script will be executed, and your firewall will contain the rules it set. Notice that I designed my script to write messages to /var/log/syslog so that there is some record of it’s activity in syslog for me to review on this headless machine.

Ubuntu 18.04 Chronicles: Static DNS settings without Stub Resolver

The “new” way to configure your network in Ubuntu 18.04 Bionic Beaver is to use netplan files in /etc/netplan/ instead of the age-old /etc/network/ .

For some reason, /etc/network still exists and you get no warning that whatever you specify there will be ineffective.

Personally, I renamed the single file in /etc/netplan/ to something that has no bearing with “cloud” and then specified my preferred network configuration. I even removed cloud-init . Here is an example:

root@sol:/etc# cat /etc/netplan/55-network-interfaces.yaml 
network:
    ethernets:
        enp3s0:
            dhcp4: true

        enp4s0:
            addresses: [192.168.1.254/24]
            dhcp4: false
            gateway4: 192.168.1.1
            nameservers:
                addresses: [8.8.8.8,8.8.4.4]
    version: 2
    renderer: networkd



That sets one interface to DHCP, and the other one to a static IP address (there are two interfaces on this machine called enp3s0 and enp4s0).

Then you need to execute sudo netplan apply and that should apply your new configuration. It does apply except that there’s one potential catch: If you’re not using the built-in systemd stub resolver, then things don’t quite work (/etc/systemd/resolve.conf with DNSStubListener=No) since the /etc/resolv.conf file is a symlink to:

/run/systemd/resolve/stub-resolv.conf

To fix this, I modified the symlink to point instead to the file that netplan automatically updates: /run/systemd/resolve/resolv.conf

sudo mv /etc/resolv.conf /etc/resolv.conf.orig
sudo ln -s /run/systemd/resolve/resolv.conf /etc/resolv.conf

Then the correct dns servers will be fetched from netplan whenever you execute netplan apply.

 




							

Ubuntu 18.04 Chronicles: removing cloud-init

You just deployed yourself a fresh copy of Ubuntu Server 18.04 Bionic Beaver. It should be the latest and greatest, and you just need a virtual machine to do some web development or perhaps you just want to enable IP forwarding and use this machine as a router. That’s great, except the latest Ubuntu assumes that you are part of the current trend to put everything in the cloud, and so ships with something called cloud-init.

No harm normally, but this wastes valuable seconds doing something you don’t need if you’re not in the cloud. It’s easy to remove this package by following the (modified) instructions here: https://makandracards.com/operations/42688-how-to-remove-cloud-init-from-ubuntu :

  1. dpkg-reconfigure cloud-init
    1. Then deselect all the options except None
  2. sudo apt-get purge cloud-init
  3. sudo mv /etc/cloud/ ~/; sudo mv /var/lib/cloud/ ~/cloud-lib
    1. I prefer to move, rather than delete, in case something goes wrong and you wish to restore the files.

When you remove cloud-init following those steps, your machine stops booting and there is apparently a service that is waiting for network to be up. This would normally be just an inconvenience, but the boot hangs indefinitely waiting for said network. Odd choice of configuration out of the box, but anyway, you can fix this by:

  1. List the services which depend on network being online.
    • sudo systemctl show -p WantedBy network-online.target
  2. This will list the culprits as some iscsi services that you probably don’t need.
  3. Disable the services
    • systemctl disable <service name

That should do to get the system booting without some service waiting endlessly for a network connection.

How To Run Need For Speed Most Wanted in Fullscreen

In a bout of nostalgia, and seeing as the game was now reasonably cheap on PC-DVD, I got myself a copy of EA’s 2012 (personal favourite) Need for Speed Most Wanted for PC. They didn’t port this title to PS4 of Xbox One.

Amazon provided prompt delivery in their usual style and I was up an running the following night with only one problem.

It was running terribly slowly on my 4K monitor using the not-so-high-end Nvidia GTX 960. The reason this was so was that the game had launched itself in 4K resolution with all the graphics settings jacked up to the max, and was running in a very large window as a sort of mock full-screen.

Not nice. I went into the game’s settings and set the resolution to 1920×1080, leaving graphics at max quality still and ended up with a tiny 1080p window in the corner of my screen. Grrr!

No option in settings to put this into fullscreen mode and scale it appropriately. Googling this led to an odd article from 2012: https://amittoor.blogspot.co.uk/2012/10/run-nfs-need-for-speed-most-wanted-in.html which suggested a registry edit which totally didn’t work, but it did trigger a memory I had of the graphics settings on my GPU.

Basically, I’ll put the steps down here for ya:

  • Open nVidia control panel (or your GPU’s control panel)
  • Expand the Display subtree and select Adjust desktop size and position
  • Select the display you desire if using multiple displays.
  • Select “Perform Scaling on GPU”
  • Tick the “Override the scaling mode set by games and programs”

That should do the trick. It appears that EA programmed the game to always try to do scaling on the Display or in software, which probably made sense when the game was being built as home users probably wouldn’t have 4K or higher resolution displays.

 

Android TV Series: Samba Services wants to track your viewing, and you can’t kill it

The latest instalment in life with an Android TV. The picture above says it all. Samba Services is an app on the TV that Sony uses to analyse what you’re viewing so that it could build a marketing profile for ad targeting. You can disable this service like most non-system services in the apps list, and you can also execute the setup of the app and opt out of the tracking.

You would think that if you’ve opted out of the service, and then gone to the apps manager and disabled the service, that would be the end of it… Wrong! You apparently can never stop the app from running. When you disable it, you get an infinite loop of popups as pictured above.

Searching the web, you wind up at a Sony Forums post: Samba Services Manager has stopped . The only way to get rid of the message is to re-enable the service.

Day by day, android TV gets me closer to becoming a believer of the Apple way.

Tracking Internet Data Usage – Advanced Tomato Style

A not oft discussed topic in router reviews (which are a dime for a dozen on the web) is how  to see just how much data your household is consuming.

Here I attempt to fill that gap by outlining the capabilities that my my router which is powered by Advanced Tomato provides. This is not a general review of the router software (which is excellent), but just some screenshots and some discussion about the bandwidth monitoring capabilities so that you can look before you leap, if you’re in the market for something that provides you with good statistics.

The router software provides two primary views by which you can monitor bandwidth usage. They are called “Bandwidth” which tracks total usage of the device across all it’s interfaces, and then there is “IP Traffic” which is useful if you want to get per-connected-device granularity.  We’re only going to cover the bandwidth monitor, but you can be confident that the per-IP statistics are just as granular.

Realtime Bandwidth Usage

Realtime Bandwidth Page

 

The realtime bandwidth page provides an instantaneous view of all the traffic going through the device. I personally struggle to make much sense of the way the view is subdivided. The LAN (br0) is the bridge that links all the interfaces on the device (the two Wifi Radios as well as the 4-port gigabit ethernet switch — I’m using an Asus RT-AC56U).

You’d expect whatever is shown in the WAN tab to be what’s actually leaving your local network onto the wider internet. WL (eth1) and WL(eth2) reflect the 2.4Ghz and the 5GHz wifi radios. It’s unclear what Eth0  represents, but between Eth0 and Vlan1, one or both of them represent the 4-port ethernet switch present on the device.

Last 24 Hours Bandwidth

24 Hour Bandwidth chart

The 24 Hours Bandwidth chart is a little more interesting. Shown above is the WAN coverage. This presents a 24-hour view of the internet. The Y-axis shows the peak average speed and the X-axis represents time of day. Shaded areas represent times when data was being transferred. Grey shades represent downloads, blue-ish shades (which are hardly visible) represent uploads.

You  can see that large file downloads will represent comparatively wider “mountains” on this graph, and the faster the download went, the taller the mountain will be. A small, fast download will be a very thin and tall mountain.

Daily Bandwidth Usage

The total transfers done per day are detailed here, going as far back as your router storage permits. This is an aggregate of internet  bandwidth usage across all devices over the date measured.

As you can see in the chart above, we did a lot of downloading/streaming on  2018-01-05.

A similar view is provided for the weekly bandwidth chart.

Monthly Data Usage

Monthly data usage

The most interesting chart to me is the monthly data usage. It allows me to see how much data we consumed going back as far as I want, grouped by month. The top-billed month would have been October 2017 if we didn’t have an unlimited Internet Connection.

There aren’t many routers out there that can provide this kind of information. The closest competitor I have found is Google Wifi, whose statistic go back at most 60 days. This is partly the inspiration for this post. I’ve been evaluating migrating to a multiple access point system in order to improve coverage in certain parts of the home, and it’s very difficult to find out how modern-day competition compares to this configuration in terms of usage statistics.

Certbot apache plugin missing

Could not choose appropriate plugin: The requested apache plugin does not appear to be installed
Attempting to renew cert from /etc/letsencrypt/renewal/nucco.org.conf produced an unexpected error: The requested apache plugin does not appear to be installed. Skipping.

When this happens, all you need to do is install “python-certbot-apache”.

Migrate from one Gmail Account to Another, 2017 edition /1

Email and Contacts

Migrating from your teenage Gmail account with the unprofessional, or hard to share email address to a new spiffy, professional email address is really easy, but scarcely documented. Here is how to do it.

Sign in to your new Account, and go to Settings (the cog wheel):

Screenshot of how to get to Settings

Then go to Accounts and Import, and select Import from another address.

From here, you need only sign in to the old Gmail address and Google will take care of the import in the background.

screenshot of accounts and imports tab with the import from another address button highlighted

You need to make sure that you have enough storage in your account for both the new and the old data. For me, this meant ensuring that I had the $2 a month 100GB plan for Google. Totally worth it. I just wish they had a 10TB option for $9.99 a month so I could use it as my cloud drive to rule them all 🙂

Enjoy.