Two interns-turned-employees share their experiences working at Groupon in both Engineering and Data Science.
What have you both been working on?
Sunil: I work with the Ranking and Personalization team in Palo Alto and we’re responsible for mining deals that are relevant to the customer. When you open the Groupon app on your phone or visit the desktop site, what you see, how you see it, and the order in which you see content is determined by the recommendation algorithms developed by this team. Every deal that shows up has to pass through a series of features, each of which does some processing, prediction, estimation, and finally assigns a score to the deal. The scores assigned by these multiple features are then combined to form a final score which determines the ordering of that deal on your homepage. Of course, there needs to be some way to make sure that the feature is actually improving user experience and is working for the benefit of the organization. And who would be a better audience to judge improvement in user experience than our users themselves? This is where the idea of live experiments fits in. All the features have to pass through an experimentation phase before they can be put into production.
Just like the deals, every user request also passes through a series of layers called “orthogonal layers.” This is the place where experiments are set up. When a user request passes through a layer, it decides what treatment the user will receive for that particular layer. It can be either the control treatment (baseline treatment which is running in production) or any of the experimentation treatment set up in the layer. The success or failure of an experiment is determined by comparing its performance with that of the control treatment. The percentage of traffic that will receive any particular treatment is adjustable. So usually an experiment starts with, say, 10% and then if results seem promising, it is ramped up to 50% and then finally to 100%. The most amazing part of the entire process, though? You get to know the approximate impact in the actual dollar amount of your individual work on the organization’s revenue. How cool is that?!
Nicholas: Pretty cool! I am also on the Ranking and Personalization team and I work on evaluating the performance of experiments and other changes to the ranking code. I own the performance testing tool which builds, deploys, simulates website traffic, and creates reports for the results. There are so many different orthogonal layers and experiments that all need to be compared, but there are simply too many to include in just one report. To make matters more complicated, each experiment has hundreds of metrics that all need to be considered to decide if an experiment is safe to add or ramp up. During my internship and my first two months as a full-time developer, I worked on building a self-service dashboard for scheduling and queueing performance test runs. Now that the dashboard is functional, I’m working on collecting runtime data with instrumentation and building infrastructure to support searching and visualizing all of the data we have available.
What are your favorite parts about working at Groupon?
Sunil: The flexible work environment and unlimited PTO policy are the top reasons I love working here. Of course both free lunch and paid lunch hours help, too. But it’s not just about the flexibility in free time, it’s also about the flexibility in the work that you do as a part of the team. There is freedom in the kind of work that you want to do, the projects that you want to work on, and the ideas that you think should be explored.
Nicholas: My favorite part of Groupon is the scope and influence I have on the projects I work on. Since every code change needs a performance run, I’ve had the chance to work with most of my team members to schedule custom runs and help with debugging. I love being able to support all of my team members, and my work helps all of them to be more efficient. I appreciate being able to decide what tasks are the most important to meet the needs of my team. Also, the proximity to the Caltrain station is so convenient; the station Sunil and I both get on is next to the Stevens Creek trail, so I get to bike along the trail to work every day.
What do you think you’ll both gain as new grads?
Sunil: There is a lot to learn about how the infrastructure is set up to manage a huge service such as ours: a lot of different technologies and languages that are used within the organization (two new languages already in my first two months) and the coding guidelines which are checked to ensure clean, sustainable, and functional code…especially for somebody who just graduated and is used to all sorts of nonsensical variable naming and structuring. There is also a lot to learn about how to build upon any ideas that you might have in mind and the ability to think out the entire process of transforming an idea to a feature in code.
Nicholas: Groupon is a great place to work because it has the resources of a large company, but there are so many important projects that a new grad like me can work on. I have encountered all sorts of challenges that need to be dealt with when scaling a small project to a reliable service. Many features that seemed trivial to implement at first required me to learn about different designs and frameworks that I never needed when my project was small.
What has your biggest challenge been so far?
Sunil: To be honest, I felt a bit intimidated when I first saw the enormity of the code base. This was during the middle of my internship when I had completed data analysis and offline model training and had to start getting my hands dirty to move the model online. I was running short on time before my internship would end because it had taken more time than planned to build a model due to some infrastructure as well as data sparsity hurdles. It took me quite some time—days, really—just to determine a starting point. After a couple of days of staring at this overwhelming code—and of course, with some help—I figured out where to start. I do think that staring at and absorbing the code inevitably helped because things started making more sense and I was able to launch an experiment with my feature just before my internship ended.
Nicholas: I was also intimated by the size of the code base when I took over the performance testing tool. I made sure to carefully test all of my changes before merging them in, and I recently decided to break the performance testing tool apart from the deployment tool. Now that the performance testing tool has become so important to us, there were many changes that needed to be made to make it reliable, customizable, and fully self-service. It was challenging to make all these large changes while keeping the service running for my team members to use; it’s hard to schedule downtime when everyone has changes they want merged!
Most memorable experience so far?
Sunil: The time I found out that my first feature was about to be rolled into production was awesome. Right before my internship ended, I had just launched an experiment with the feature that I was developing at 20% traffic. When I joined full time, my first question to my team was, “how did it go?” And they let me know it went well. Later on, I saw the performance charts and then there was a presentation from the product team where they gave the exact million dollar effect that my feature had on revenue. That was quite memorable for me and satisfying to see the impact of my work on a large scale!
Nicholas: The holiday season is a tough time for most e-commerce companies, and the performance testing tool is very important for evaluating changes for holiday readiness. My most memorable experience was when my manager called me after Black Friday and told me that the Ranking and Personalization service was able to handle all the traffic we got as a result of my hard work. The tool allowed us to create a stable holiday readiness branch and made debugging performance issues much easier.
And finally… what drew you both to come back to Groupon as new grads?
Sunil: For me, it was really the kind of work that I had done during my internship. It was closely related to the specialization that I was pursuing in my Master’s degree. It was pretty much exactly what I wanted to do further in my career.
Nicholas: I also enjoyed the work I did during my internship and I’m looking forward to applying some of the research I did as an undergraduate to allow for faster and more reliable performance run results. I chose to continue working at Groupon because it’s the best place for me to improve as a developer and it’s a place where I can have a large impact in my company.