Lessons Learned
Troubleshooting results, failed attempts, technical takeaways, and next steps from building and maintaining Headley Lab.
One of the biggest lessons from the lab has been to avoid changing multiple things at once. I try to isolate the problem, test one layer at a time, make a controlled change, and verify the result before moving on.
Plan → Build → Test → Troubleshoot → Verify → Document
Commands, logs, packet captures, service status, firewall rules, and repeat testing provide better evidence than assuming a configuration change worked.
I learned that successful ping tests do not prove DNS is functioning. A client can have valid addressing and network connectivity while name resolution, authentication, or application access is still broken.
A consistent troubleshooting order makes network problems easier to isolate:
Addressing → Local connectivity → Gateway or route → DNS → Port → Service → Firewall → Application
An external USB hard drive connected to BatPi was intended to serve as a local backup destination. The drive intermittently failed to initialize and occasionally caused Raspberry Pi boot and stability problems.
I checked USB devices, storage devices, mounted filesystems, and kernel messages:
lsusb
lsblk
sudo fdisk -l
df -h
dmesg | tail -50BatPi operated normally when the HDD was disconnected. Reconnecting the drive reproduced the issue, isolating the problem to the external HDD and its power requirements.
The mechanical hard drive required more power during spin-up than the Raspberry Pi USB connection could reliably provide. This caused intermittent initialization and unreliable backup operation.
I moved the HDD backup workload to a Windows system with more reliable power and storage connectivity.
BatPi continues to provide:
I verified the storage device in PowerShell using:
Get-Disk
Get-Volume
Get-PhysicalDiskBackup files were then checked to confirm that data was successfully written and accessible.
This issue reinforced the importance of isolating hardware problems before changing software configuration. A system may support USB storage while still being unable to reliably power a mechanical hard drive. Moving backups to Windows provided a more dependable solution while allowing BatPi to focus on network services.
VLAN segmentation is most useful when communication between networks is controlled deliberately.
I configured an IoT VLAN with restricted access to the primary LAN while allowing required DNS traffic to the Pi-hole server over TCP and UDP port 53.
During Pi-hole testing, a Windows client could successfully ping the server over IPv6 while DNS queries still timed out.
ICMPv6 connectivity proved addressing and Neighbor Discovery were working, but it did not prove TCP or UDP port 53 was permitted.
Working with IPv6 required understanding prefix delegation, Router Advertisements, SLAAC, RDNSS, ICMPv6, Neighbor Discovery, and host firewall rules.
I used tcpdump to confirm that IPv6 DNS requests were reaching the Linux server.
Because the requests arrived but no DNS response returned, I narrowed the problem to the server rather than the router or switching path.
Port testing then confirmed that TCP 53 was blocked. Missing IPv6 UFW rules were added, and DNS began working.
I learned that Active Directory authentication, Group Policy, and access to domain resources can fail even when a client has basic IP connectivity.
Hyper-V checkpoints provided a safe way to test configuration changes, troubleshooting exercises, and recovery procedures without rebuilding a virtual machine every time something went wrong.
I practiced the difference between local and domain accounts and learned to verify which security context a user is operating under before troubleshooting permissions or access.
Commands such as systemctl status, logs, network listeners, and firewall rules provide useful evidence before restarting a service.
I learned to distinguish commands that show interface state from commands that actually change it.
When SSH, RDP, or PowerShell Remoting does not work, I verify reachability, listening ports, firewall rules, service status, and user authorization before assuming the remote-access tool itself is broken.
Administrative access should be used only when necessary.
I practiced removing unnecessary elevated access and verifying security settings rather than leaving accounts with more privileges than required.
Malware removal does not always restore confidence in a system. Depending on the severity of a compromise, a reset, reimage, or clean installation may be safer.
Restore points can help recover certain system changes, but they do not replace proper backups, restore testing, and recovery planning.
Not every lab ended with a clean fix. I keep unresolved problems in my documentation because the troubleshooting process is still useful.
Investigated Windows Sandbox failures involving errors 0x80070003 and 0x80070002 by reviewing Windows features, virtualization services, system files, container components, and graphics-related configuration.
Troubleshooting isolated the problem to the system's physical GPU configuration rather than Windows Sandbox itself.
After correcting the GPU-related setting, Windows Sandbox launched successfully and operated normally, confirming the graphics configuration as the root cause.
The issue reinforced the value of isolating hardware and software dependencies, changing one setting at a time, and verifying the result before continuing with additional troubleshooting.
Investigated Windows activation and virtual-machine licensing issues and learned to separate licensing problems from networking, operating-system, or application failures.
Current lab work includes virtualization, Windows and Linux administration, Active Directory and DNS, Raspberry Pi server administration, managed switching, VLAN segmentation, Pi-hole DNS filtering, IPv4 and IPv6 networking, wireless infrastructure, scripting, security exercises, backup and recovery, and network troubleshooting.
Detailed technical project work is documented on the Projects page.
View Projects View Project Pictures