-
Git Recombine Pattern for /etc config
Recording
/etc/config in git is a recommended way to track history and revert breaking changes. In many cases, content needs to be imported and transformed from other repositories before being deployed as/etc/config on a VM . Config changes to/etcare often made in haste during development & in emergencies. While git is helpful in recording those changes locally, often content needs to be managed on one machine and pushed to another. Or content can be in an outside repo with a different directory schema. -
Why GCP is More Usable for Developers
Teams should consider many qualities when choosing a cloud provider like AWS, GCP (Google Cloud Platform) or Microsoft Azure. Product offerings, familiarity, pricing, and usability – among others.
Compared to AWS, Google Cloud Platform (GCP) is more usable for developers due to it’s core design approach surrounding resources, projects, APIs and Identities (IAM). This project-first approach avoids common bad practices like spaghetti namespaces, excessive permissions, and accidental exposure. Moreover, GCP includes much more advanced logging & alerting tools, comparable to Splunk and Data Dog, right out of the box.
-
Fighting GCP & Firebase Cloud Client CLI and SDK Bloat
Client CLIs & SDKs for GCP, Firebase and other clouds are terribly bloated. GCP includes a python distro, firebase includes node+npm. This goes unnoticed on overpowered devboxes, but impacts your cloud bill with storage, vcpu, wall-time and transfer fees. If you are trying to downsize your VMs, you will find that the client SDK/ CLI pre-requisites will often hang your machine terminal by exausting vcpu and iops budgets. Cloud container services are often storage-limited to ram-disks–so CLI installs consume what little you have.
-
WSL2 Backup to OneDrive Cloud
WSL2 provides great disk performance, but it requires storing the files separately in a virtual disk that is not accessible by OneDrive. WSL2 can be backed up with wsl –export Debian to a VHD or TGZ, but that is a complete disk backup of 20gb or more – not scalable for hourly backups.
With this approach, we use Windows Task Scheduler to trigger
robocopyto incrementally sync directories from WSL2 to Onedrive’s native FS, so incremental copies are fast ( 1 s per 10k files), and OneDrive sync time remains negligible. -
IPV6 Migration Guide for Developers using AWS EC2 -- A Primer
With the news that AWS will be now charging about $4 / instance-month for public IPv4 addresses, many developers who procrastinated ipv6 migration are finally updating both ends of their development setup.
It’s a great time to migrate, as all the intermediate infrastructure now supports IPV6 readily. Moreover, you’ll benefit from permanent , global addresses for your development instances.
Pros
- A single, global, stable address for EC2 instances that never changes. No need for dynamic DNS and other hacks
- No need to pay for Elastic IP addresses on dev instances
- Global addressing for mutual duplex services (no more NAT needed)
- Better flexibility and clarity for addressing, including Link Local & local addresses
Cons
- Time needed to migrate infra to IPV6
- Clumsier & less-memorable addresses, with unfamiliar idioms (e.g. no more using 127.0.0.1 or 192.168.1.1 – though there are replacements) inherent
- Bugs in legacy code that assumes 32 bit & string-representations of ipv4 addresses
Concepts & Approach
In theory, IPV6 uses 128bit addresses in place of 32 bit. Most of the intermediate infra (ISP, backbone) is now compatible. The two areas of attention for developers would be the server side with AWS , and the client side with your home/office network.
-
BeeLink SER6 MAX Out-of-Box Bloatware / Spyware / Malware Review
I recently set up a new SER6 and reviewed bloatware / spyware / malware prior to connecting to the internet. There were quite a few posts asking about spyware, and given it’s competitive price point, I was also a bit suspicious. I usually do a malware review before connecting any new device to the internet.
What I reviewed
- running processes and their signatures
- startup apps (Task Manager formerly msconfig)
- Installed Services
- Windows Features Enabled
- Partition Table Review (for malware)
- Local user accounts
- Confirm installed hardware components and brands met specifications.
- BIOS & Windows 11 Secure Boot, TPM & Enhanced Hardware Security settings (see Questionable)
tl;dr & verdict -- This Beelink is the cleanest windows machine I’ve purchased (including Dell, HP, Alienware). Nothing installed would be considered bloatware , spyware or malware. A few installed options (see Questionable, below) were probably added for user-acceptance testing.
-
Improve WSL Security with Read-Only Filesystem
By default, all Windows drives are mounted with read & write access (rw) within WSL . Though this is convenient for beginners, it opens up VM shell attacks on your Windows host files.
Instead, we can disable the auto mount feature using
wsl.confand selectively add read-only drives inside the WSL VM using/etc/fstabOverview
- Deactivate “auto mount” in
/etc/wsl.conf - Enable fstab using
MOUNTfStAB = trueinwsl.conf - test config files and mounting work well
- reboot the wsl VM to complete the setup
Example WSL Config
wsl.confPlace this inside the /etc/ directory on the WSL VM
- Deactivate “auto mount” in
-
Smokeping On Raspberry Pi Zero
Smokeping is a self-contained network monitoring app , capable of monitoring using ICMP/Ping, HTTP, DNS – as well as other signals generated from CLI monitoring tools (e.g. curl, dig, mtr etc). It provides a web-based monitoring UI to chart the probe measurements so no further monitoring apps (like Prometheus) are needed.
Running smokeping on a $5 Raspberry Pi Zero is a fun experiment in lightweight computing . Using Apache Mod FastCGI makes the app usable on the meager hardware.
-
Testing Without Excuses
Every app has that last inch (or mile) of code that’s not covered by tests. Usually it’s an interactive cycle of compile-run-inspect on the command line like
You Test
curl -X POST https://reqbin.com/echo/post/json##👀 You Expect:
{"success":"true"}Despite having 3-4 testing frameworks for unit tests, e2e, regression etc– there’s always a gap where you find yourself re-playing commands in the terminal to test.
A common case is 🔥firefighting where ad-hoc tests are needed to validate an emergency config change or deployment.
-
Three Pillars
Recently an old friend, with great experience as an IC, PM and EM, called me to ask for some advice. He had been running his business for a while and took up a new role as an engineering manager after some time. “What areas do you focus on as an EM?, particularly when joining a new team”.
I divided the conversation into three pillars: strategy & inventory, technical (aka going deep) and career / personal
-
Signal Vs Noise
One responsibility of engineers & especially leads is managing many channels of signals : emails, blog posts (internal and external), tags , push notifications, group chats, alerts from dashboards and more.
These signals tend to scale exponentially to the number of projects & people that you are responsible for.
Quickly you’ll need to set up a system to make sure that you are receiving high-signal information and filtering out low-signal noise. How do you do that?
-
A Timeless Directory Layout for All of your Projects
Directory layouts are like log cabins that start from a basic shed, gradually adding a room at a time. When you start out on UNIX, everything gets thrown in your home directory. Over time you start to develop a structure for your sources, binaries, projects, data files (like CSV, images, tar files), config, etc
My layout is called TDL – because it allows me to juggle open source projects, partnerships and jobs in a consistent structure across machines and time.
-
Snooze to Save Money
Cloud instances bill by the hour (or the minute) – and right now you’re burning money. Use
snoozeto auto-shutdown your instances in 45 minutes.Add
snoozeto your~/.bashrcalias snooze='sudo shutdown -c ; sudo shutdown -h +45 &' snoozeWhen you want to extend your session, run
snoozeBroadcast message from ec2-user@ip-172-31-43-250 (/dev/pts/1) at 2:50 ... The system is going down for halt in 45 minutes!How does this work?
shutdown -ccancels the shutdown, andshutdown -h +45schedules a shutdown in 45min. -
Fully Remote Development with VS Code & Cloud9
I work from about 7 different machines, including 3 laptops, ipad, chromebook and a PC desktop. Usually this means keeping credentials, config, build dependencies and IDEs in sync across all 3–and the iPad & Chromebook just can’t run my dev environment
I considered a few options to enable seamless work across devices
option pros cons Keep a “dev” docker image that contains everything. fully-local dev only works on Desktop OSs. Inconsistency if you forget to push the image Sync script fully-local dev Inconsistency across devices. Script mayhem Code remotely via a VM Secure, consistent Traditionally, text-only Solution
- Launch Cloud9 Environment on AWS
- Install VS Code + SSH Remote Extension
- Install tmux
With this setup, you get the highest-fidelity experience when you can (with VS code), plus an adequate experience on iPad & Chromebook (cloud9 web ide). With tmux you get seamless handoff across all devices.
-
Build 100kB Docker Images from Scratch
📓 The Gist
You may think your 100mB Alpine images are small–but how about 100kB? Smaller images ship more quickly, and contain fewer attack vectors. Moreover, by optimizing images, you discover and isolate exactly what is needed for your app to run.
Let’s Optimize.
There are two key characteristics of scratch-based docker images:
- The Dockerfile has two build stages:
- a builder–which contains all of the build dependencies including source, libraries and tools and..
- a final image, containing the binary and any run-time dependencies (config files, certificates and dynamically linked libraries)
- The final image is
FROM scratch– the empty docker image
With this approach, your run-time image will contain exactly what is needed for your app to run – no additional config files, daemons or libraries that could be misconfigured or exploited.
-
Publish Free Static Websites With Firebase, Hugo and Google Cloud Builder -- Part 2
In Part 1, we completed our development environment, including setting up Hugo and our repo.
Here we’ll publish our site to Firebase Hosting, and create the CI tools on Google Cloud Build to build and publish upon push.
Open Your Cloud Shell
In Part 1, we enhanced our cloud shell with hugo and set up our repo. In Part two, we’ll use it to create the builder and configure hosting.
See the Quickstart for complete instructions
-
Benchmarking Pihole : Pi Zero vs Pi 3b+
Here’s a benchmark comparing pi-hole running on a Pi Zero (with USB ethernet) vs a Pi 3b+.
tl;dr There was negligible performance difference for blocked domains, but a measurable difference in mean for forwarded + cacheable domains. Although the Pi 3b+ has a 11ms better mean response time for forwarded queries, the P95 for pi zero is better in both blocked and forwarded queries.
I would recommend using the Pi Zero.
-
PHP Dev Environment One-Liner
Here’s the fastest way to get your PHP app running. No MAMP, WAMP, apache or any of that nonsense.
Moreover, it allows you to run multiple projects independently.
I’m assuming you have docker.
tl;dr
This runs the php docker image, mounts the current directory, and spins up a server on port 8086
$ docker run -v $(pwd):/www -it -p8086:8086 php:5.6-alpine sh -c "cd www; php -S 0.0.0.0:8086"The Full Version
Create your index.php
$ cat > index.php <html><body><h1><?php print("Hello World!") ?> </h1></body></html> CTRL-DRun the Server
$ docker run -v $(pwd):/www -it -p8086:8086 php:5.6-alpine sh -c "cd www; php -S 0.0.0.0:8086"Test Your Server
$ curl localhost:8086 <html><body><h1>Hello World! </h1></body></html> -
Being Scientific with Gists : The Sharable Laboratory
Next time you create a post with code snippets–like here on dev.to or stackoverflow–consider sharing a working and buildable gist along with it. By doing so, others can clone, reproduce your results, and commit new variants much more easily.
With the process below, your gist becomes a sharable laboratory. Since the gist contains all of the code variants and test cases, any team member can create a variant and run the tests against all existing variants.
-
Getting to Yes -- As Quickly as Possible
There was a great discussion a year ago about how fast gnu’s version of “yes” is. If you’re unfamiliar,
yesoutputsyindefinitely.yes |head -5 y y y y yThe key takeaway was that
writeis expensive and writing page-aligned buffers is much faster. The is true across languages, so let’s see how to do it properly in go.If you’re shocked or impressed by the results, let’s see you do it in your language – post your results in the comments.
Tony Metzidis
Software Developer