Work History
Ditto.live
July 2023 - Present
Engineering Manager, SRE
Ditto is a casually consistent distributed database utilizing CRDTs. Through a Rust SDK we are able to provide peer to peer communication and local first development for web and mobile applications.
Accomplishments
- Managed 12 engineers developing custom kubernetes operators that managed our distributed database.
Zapier
March 2021 - July 2023
Engineering Manager, SRE
Zapier is a no-code solution for automation. It ties many APIs together through a combination of webhooks and polling.
I was the engineering manager of the largest SRE team in the organization that covered americas timezones to asia pacific (APAC).
* Monitoring and Metrics through a mix of Grafana, Prometheus, DataDog, PagerDuty, and Graylog.
* Ran multiple elasticsearch clusters
* 1:1 with my direct reports
Accomplishments
- Managed 12 Kubernetes cluster for both production and staging environments.
- Manage an elasticsearch cluster that received 7tb of data each day
- 100% Infrastructure as using Terraform, Helm, and GitLab pipelines.
- Built the primary GitOps tool for managing which images are deployed to each environment. This included monitoring ArgoCD statuses and Slack Notifications for each team.
- Helped run the migration from GitHub to GitLab which included migrating thousands of repositories and CI/CD pipelines.
- Ran upgrades of Kubernetes from 1.16 to 1.24
- Built training and onboarding documentation for new SREs
- Scrum ceremonies, Quarterly planning, and Performance reviews.
- Designed and implemented the interview process for SREs.
- Designed and implemented the onboarding process for SREs.
- Introduced and rolled out our entire FinOps strategy and cost management.
- Improved our SOC2 policies and management through better processes and understanding of SOC2 controls.
- Built out a better monitoring and auditing system for our production bastion access.
Stacklet
March 2022 - June 2023
Engineering Manager, SRE
Stacklet is a cloud governance and FinOps tool that allows organizations to main their SOC2, CIS, and ISO certifications through a set of policy packs and a cloud asset inventory.
It is built on top of the opensource tool Cloud custodian.
I was the engineering manager of the team developing some of the most critical parts of the application such as:
* Cloud Asset Database
* Compliance Notifications via Slack, Email, and JIRA
* FinOps and Cost Analysis
* Shift-Left compliance through terraform policies
* This included the management of all the infrastructure being deployed. The infrastructure was 100% Infrastructure as Code utilizing terraform with a mix of Github Actions and AWS CodePipeline.
Accomplishments
- Introduced the AWS Cost and Usage Report functionality which moved us into the FinOps space.
- Managed the rollout of both GCP and Azure support for our asset database
- Introduced the SLI/SLO framework and rolled out the new monitoring infrastructure that all teams utilize.
- Revamped the on-call process and introduced a throughout retrospective system for incidents.
- Along with our SaaS offering we also had an on-prem offering so our IaC had to work flawlessly to spin up new accounts on the fly.
EventRay
June 2016 to March 2021
Engineering Manager, SRE
EventRay is a conference registration platform to help meeting planners easily build their registration website, collect payment for the event, and analyze the attendee data to make better decisions while planning the on-site.
* Built the infrastructure team from scratch growing it from a start-up to more production ready infrastructure that was SOC2 and CIS compliant.
* Migrated from EC2 instances that were provisioned with ansible to Kubernetes, ArgoCD, and GitOps.
* Moved monitoring stack from nagios and hand made python scripts to Prometheus, Grafana, and Thanos.
* Our infrastructure was a multi-cloud environment built on a combination TeamCity, Github Actions, Ansible, Docker, AWS, and our own half-rack data center built on top of VMWare and Proxmox.
Accomplishments
- Developed our own embedded programming language for declaring logic on forms for hiding/showing elements, changing responses, and navigating to different pages. Wrapped this in an easy to use UI so that our customers could create truly dynamic registration forms.
- Developed a cross-platform internationalization (i18n) engine using gettext so that all our applications (React.js, React-Native, Electron) and our APIs (Python) could share translations.
- Developed a data analysis engine using the same embedded language and UI that the logic for our registration forms used to provide a truly ad-hoc reporting engine
- Scaled out to 15,000 requests per minute for a single event
- Collected over $30 million dollars for our customers through a variety of payment gateways (Payflow Pro, Stripe, Authorize.net)
- Integrated with Salesforce, Zapier, MailChimp, Google Suite, and many more industry specific API providers.
- Intefaced with USB HID devices to do 1D, 2D, and QR barcode scanning.
SurveyMonkey
March 2013 to June 2016
Principal Software Engineer
SurveyMonkey is a self-service survey and response collection engine. They are a service oriented architecture with 47 projects deployed over 200 machines and serve millions of requests per hour. They are developed in Python (Pyramid) on top of memcached, redis, and SQL Server.
I worked on the analytics team where I developed the tools for survey creators to get insights on their response data and create ad-hoc reports in real-time and also export to multiple formats, including XLS, PPT, and CSV.
I lead the team that built the new generation of analytics engine based on Kafka, Zookeeper, and Cassandra that powered our polling team for the 2016 presidential election that was covered by CNN, New York Times, and MSNBC.
Accomplishments
- Architected the central logging, caching, and validation libraries that all services within the company use.
- Designed a new real-time ETL engine that extracted terabytes of data from SQL Server 2012 and transformed it into 2D arrays that were encoded in a sparse matrix with variable length data types that tremendously reduced the size of the data and overall query time.
- Automated the deployment of all 47 projects and provisioning of the application servers using Ansible. These scripts were also used within vagrant to provision local development stacks.
- Helped with the initial adoption and deployment of openstack within the company
- Helped improve performance and overall scalability. This includes improving our caching and migrating us to gunicorn from paster and optimizing nginx.
- Designed the gunicorn strategy for zero downtime deployments for continuous deployment
Devtoo
December 2011 to March 2013
Senior Software Engineer
Works as a software development contractor where I contributed to a variety of companies and platforms.
I worked on Python, C#, and PHP projects as a full-stack contractor. On the frontend I worked on jQuery, Backbone.js, and React.js applications.
Accomplishments
- Developed an online classroom (mooc) that integrated a PHP LMS (moodle) with google classroom technology.
- Built a real-time chat platform utilizing RabbitMQ and socket.io
- Created an open-air photobooth that utilized canon SLR cameras and PyGame to layout branded photos at bars, concerts, and parties.
Pictage
May 2011 to December 2011
Senior Software Engineer
Pictage (ShootQ) is a business management platform for professional photographers. I helped develop web applications and services that would allow photographers to quote services, collect payment, and share the final photos with their customers.
We had a large turbogears application as well as a custom webframework we developed (PecanPy) that both utilized SQLAlchemy, Celery, Redis, MongoDB, and plenty of javascript. Each developer was in charge of the full stack of whatever they worked on.
We integrated with Amazon S3, Facebook, Twitter, and Salesforce
Accomplishments
- Worked on ShootQ, a web service used by photographers to manage their business and CRM data.
Worked with a variety of technologies to solve unique scalability problems, including varnish, redis, and RabbitMQ
- Worked on nimbus and atrium, a unified “website/blog” and “customer identity” platform used by photographers to establish online presence for their businesses.
Helped build the tools that helped their brand their business website, photo galleries, and invoices.
Orchid Events
January 2006 to May 2011
Senior Software Engineer
Orchid is a conference Management and registration software for generating dynamic forms with complex business logic. We were an enterprise event management company that had both a SaaS offering as well as an on-premise product.
I helped architect and develop every feature of the product. It was developed in C#, ASP.NET, and SQL Server.
Accomplishments
- Had to scale up to 15,000 writes per minute
- Implemented standards to make the company PCI compliant
- Created an analytics engine for filtering and executing custom reports and exporting them to a variety of forms (excel, pdf, word, html).
- Interfaced with multiple payment gateways through Authorize.net and Verisign and developed the encryption necessary to pass the visa requirements to store credit card information
- Helped manage the Cisco PIX firewall and many layer2/3 switches for both our development and production environments
- Configured and managed the subversion server setup (tags, branches, trunk) and merged over 5 years worth of code from sourcesafe and team foundation server to SVN.
My Horse Player
April 2005 to January 2006
Software Engineer
MHP was a data analytics platform for horse handicappers. We developed our algorithm for classifying horses and provided tools to our customers to predict winners based on criteria they chose.
I had to parse and analyze terabytes of historical horse racing data and generate statistics to help handicappers bet on the races.
Accomplishments
- Created a windows application using XML, SOAP web services, and C# .NET for generating and displaying statistics on horse races.
- Ported the windows application to an ASP.NET application
- Created a full company web based intranet with C# which involved task scheduling, time sheets, work orders, inventory, and sales quoting. All authentication and grouping was handled through LDAP.
- Our algorithms picked winners more often than machine learning attempts at the time utilizing the same data