How I got on the team
In 2019, the Ops team announced a hackathon open to all of CIS. A draft split 20+ volunteers into four teams. I wasn’t picked. Nobody at CIS really knew what I could do. But a former mentee did, and he told his boss/peer, Darrin, to grab me. So I joined.
What we built: ML-driven patch prioritization
The team chose threat analytics, and we landed on building something to correlate data from our vulnerability-management platform, our Palo Alto firewalls, and our network routes to prioritize patching for Nike. A teammate built a web front end and the initial pandas data-engineering. We partnered with Nike’s CTA team to train a model on the unified data. CTA, short for Cyber Threat Analytics, was led by a brilliant ML engineer. I owned the data-collection system.
My data collection, in Python. A pluggable collector framework, with a common interface each source implemented, that gathered from every source and wrote to Parquet on shared disk. I added new sources, including public CVE data: rating, disclosure date, severity, and whether a known exploit or malware existed.
Then I ended up rebuilding most of the rest, because the first cut didn’t scale:
- The data engineering. The original looped row by row before one-hot encoding, which did not scale to millions of records. I rewrote it properly with
.locand vectorized operations, which cut processing time dramatically, and added the CVE enrichment. - The web/app tier. It was single-threaded. One button on one web page kicked off collection, engineering, and analysis as a run of 16 to 20 hours that died if the browser dropped or Python crashed. I put it behind gunicorn, a WSGI server, with HTTPS, then wrote my own task-management framework in Python. It had a standard interface to register any job, a registry and run-ordering, live reporting of active and running tasks, failure capture, queueing and scheduling, guards against duplicate queueing and concurrent launches, and the ability to trigger one, several, or all steps from the UI. All of it ran independent of the web front end.
- Parallelization. On startup it reads the host’s CPU core/thread counts, splits the data engineering across workers, runs them in parallel, then collects and emits the final result. That took the 20-hour, fragile process down to a ~16-minute run bounded only by the host.
- Build/deploy. I wrote the CF build and deploy, with cert retrieval from the HSM, plus a set of aspect-oriented-style decorators for logging, authentication, and exception capture. A very fun build.
The model / scoring
The system rated every host-level vulnerability or patch on a 0 to 2 scale, where 2 meant patch now. It combined three questions. Is the patch exploitable, from host vuln data and CVE data? Is the host exposed to the internet, from NAT and routing rules? Is the service exposed, from Palo Alto firewall rules? Set values from the vuln-management team fed in. Fully automated and validated. This is the Nike instance of the “patch prioritization at scale” thesis that runs from NeoPrime forward.
Net result
It blew everyone away. Judging and awards slipped to 2020, and landed the week we were all sent home for COVID. I never returned to the office, and I left Nike five years and one day after I started. I heard Nike intended to file a patent around it. The VP of Ops handed out bottles of whiskey/scotch to the team. Six of us were on it, and I did the majority of the critical work.
What I learned
- The win wasn’t the model. It was re-seeing the pipeline. Turning a 20-hour, failure-prone single run into a 16-minute parallelized one came from picturing the whole system end-to-end and rebuilding its guts, not from a cleverer algorithm.
- Mentorship compounds. The person who pulled me onto the team was someone I’d pulled into tech years earlier.
Key people
- A former mentee Paul brought into tech, who got Paul onto the team.
- A teammate, who built the initial web front end and first-pass data engineering.
- Darrin, the Ops lead/peer who drafted Paul at a former mentee’s urging.
- Nike CTA team lead, the “brilliant ML guy” whose team trained the model.
Technologies used
Python, for the pluggable collector framework and custom parallel task-management engine; pandas, vectorized/.loc and one-hot encoding; Parquet; gunicorn/WSGI + HTTPS; CVE data enrichment; Palo Alto firewall rule + NAT/route correlation; vulnerability-management data; ML model, with Nike CTA; Cloud Foundry build/deploy; HSM cert retrieval; AOP-style decorators for logging, auth, and exceptions.
Patents / awards
Patch Prioritization at Scale. Nike intended to file a patent around it. Hackathon-recognized in 2020.