The unglamorous security wins (and the crypto-miner I let in)

One semester I built a secure distributed-computing system with a small team. The brief was deceptively simple: a web app where a user uploads a script, an API that hands it off, and a cluster of worker nodes that run it in parallel and return the result. The word "secure" was in the assignment title, which turns out to be a useful forcing function, because it makes you justify every layer instead of bolting protection on at the end. Looking back, almost everything that actually protected us was boring. And the one thing that bit us was even more boring than that.

Security starts at the architecture, not the code

The most important security decision happened before anyone wrote a feature: choosing the shape of the API. We went with a classic three-layer design. An API layer that talks to the outside world, a business-logic layer that does the actual thinking, and a data-access layer that talks to the database and any external sources. The rule that makes it a security property rather than just tidy code is strict separation: the API layer can only call business logic, it is never allowed to reach the data-access layer directly, and the layers communicate through interfaces so they don't bleed into each other.

Why does that matter for security and not just architecture? Because it makes the unsafe path impossible to take by accident. A request from the outside physically cannot reach the database without passing through the logic that validates it. You can't forget to add the check, because there is no route that skips it. A bonus is replaceability: because the layers only know each other through interfaces, you can swap the data layer or refactor the logic without touching the boundary that faces the internet. Secure design and maintainable design turn out to be the same design.

Least privilege, everywhere

A handful of habits did a disproportionate amount of the actual protecting, and they all come back to the same principle: give every component the smallest amount of power it can do its job with.

• An ORM instead of hand-written SQL. Using Entity Framework meant we worked with models, not query strings. That closes the door on a whole class of SQL injection by default, because you have to go out of your way to build a raw, unsafe query. The safe option is also the easy one.
• A scoped database user. The API connects with a database account that has permission to run only the specific queries the API needs, and nothing else. If the API is ever compromised, the blast radius is that short list of operations, not the entire database. This is the Principle of Least Privilege applied where it counts.
• Secrets out of the source. Connection strings, keys, IP addresses and the JWT configuration lived outside the code, in app settings, so they wouldn't show up to anyone reading or reverse-engineering the build. The program references them, but they aren't baked into it.

None of this is clever. All of it is the kind of thing that quietly prevents the incidents you never hear about because they didn't happen.

Make plaintext impossible, not just discouraged

Authentication is where it's easiest to get lazy, so we tried to design laziness out of it. When a user logs in, the server sends back the salt stored for that username. The browser combines the salt with the password and hashes the result with SHA-256, and only that hash travels over the network to be compared against the stored value. The plain password never leaves the device.

This isn't a replacement for TLS, and it doesn't make you immune to everything. It's a belt-and-braces habit: even if one layer fails, a single weak link shouldn't be enough to expose someone's actual password. Defense in depth means assuming any one control can fail and making sure that failure isn't catastrophic on its own.

We also used a WebSocket connection for the live progress updates from the cluster, and tied each connection to a token so the system always knew which client a given result belonged to. Real-time feedback is a UX feature, but the token is the part that keeps one user's results from leaking into another's session.

Isolate the code you don't trust

There's a special problem in a system that runs user-uploaded scripts: by design, you are executing code you didn't write on your own machines. Early on we ran workers directly on host machines, which meant the master process effectively had full access to those hosts, including the ability to read or delete files. That is a terrible amount of trust to hand to an arbitrary uploaded script.

The fix was Docker. Each worker runs in a container, so an uploaded script is boxed into an isolated environment instead of touching the host. Spinning up a new worker became "clone two files and start the container," and just as importantly, a malicious or buggy script can't reach past the container walls. Isolation turned an alarming amount of implicit trust into something bounded and disposable.

Watch what's happening

You can't respond to what you can't see, so we built monitoring on Wazuh and Elastic across three angles: performance and health, security events, and malware detection. The security dashboard mapped alerts to MITRE ATT&CK tactics, so instead of a flat list of log lines we could see categories like initial access, privilege escalation, persistence and lateral movement, each with a count and a timeline. Monitoring is the layer that turns "we hope nothing is wrong" into "we would notice if it were."

Let someone else try to break it

We ran a red-versus-blue exercise, and it was humbling in the most useful way. Most of our defenses held, which felt great for about a minute, and then the attackers found the gaps we'd stopped looking at: no CSRF token on the API, and a login cookie missing its HttpOnly and Secure flags. Both are one-line fixes. Both are invisible until somebody actively probes for them. That's the entire argument for adversarial testing, you stop grading your own homework and let someone whose job is to find the hole actually find it.

And then the boring thing that actually got us

Here's the part I think about most. To update the website we had left an SSH port open to the internet, on a server that still had its placeholder credentials, student and student. Someone found it, logged straight in, and installed a crypto-miner. It chewed through the machine's resources until our network's IP address got pulled offline while we cleaned up the mess.

Sit with that for a second. The layered architecture, the scoped database user, the client-side hashing, the container isolation, the monitoring, all of it was completely bypassed by one default password on one exposed port. The attacker didn't defeat our best defense. They strolled through our worst one. The fix was as boring as the mistake: set a real password, and make the SSH port reachable only from inside the network so it isn't sitting on the public internet at all.

The lesson

Your system is exactly as strong as its weakest default. Good architecture, least privilege, sensible hashing and real monitoring are genuine wins, but they are ceilings, not floors. An attacker doesn't fight the strongest part of your design; they look for the dumbest open door and walk through it. So now, before I admire any of the sophisticated pieces, I go hunting for the unglamorous failure first: the default credential, the exposed port, the missing cookie flag, the service that should never have been reachable. That's almost always where the real risk lives, and it's almost always the cheapest thing to fix.