Zhe “Calvin” Song – Software Development Engineering Intern in Palo Alto

Zhe “Calvin” Song – Software Development Engineering Intern in Palo Alto

ZHE “CALVIN” SONG

Software Development Engineering Intern
Palo Alto, CA
Joined: 2018

You can’t read Calvin’s internship story and NOT feel amped up about Data Science.

What school do you go to?

Northeastern University – Seattle

What team are you interning for?

Data Science Platform

What are you working on or what will you be tackling while you’re here?

I’m working with the Data Science Platform Team in Palo Alto. We are responsible for building an end-to-end self-service platform to support Machine Learning/Data Science as a service at Groupon. Imagine that if you were a Data Scientist, how excited you would be if you had a unified, standardized solution that takes care of the entire data science lifecycle for you? That’s Holmes!

My project is part of this exciting platform, which focuses on building a generic data-ingest solution for click-stream data to be surfaced in real-time fashion. Every time a customer purchases a deal, or claims/redeems a deal at Groupon, an event will be generated and processed as “click-stream” event that is of my interest! I build real-time streaming data pipelines and API to consume, process, store, and expose click-stream data with a bunch of really cool cutting-edge technologies such as Apache Kafka, Beam and Spark, Redis, JTier (Groupon’s Java tier built on top of Dropwizard).

What’s been most surprising so far about interning at Groupon?

Before I started my internship at Groupon, I was prepared to do some of the “debug-oriented side projects” with the team. I figured since I was going to be an intern, the company may not trust me enough to assign important or high impact jobs to me. However, I was surprised from the first day I joined here; my manager went through the architecture of the entire platform built in our team and set up a series of interesting and challenging goals for me.

The DSP team is a new team that just formed maybe two weeks before I joined here, which means everyone in this team are actually building the platform (Holmes!) from scratch. As an intern, I am super excited that I can contribute and make impacts on the existing project with other full-time engineers. Everyday here I learned a lot from those talent and experienced people surrounding me, from coding practices to software development life-cycle, what a wonderful experience!

What’s the coolest thing you’ve accomplished so far this summer?

I built a generic real-time streaming data pipeline that’s running on top of Apache Beam to serve as a real-time solution for all click-stream events. The pipeline is meta data driven so it’s highly extensible for clients who need to customize the schema for specific event based on their requirements. How cool is that?!

What’s the most innovative technology you’ve experienced in your lifetime?

I was a graduate student majoring in bio-analytical chemistry before I decided to switch my career to computer science. During the time as a research assistant in a bio chem lab, I got the chance to play with one of the most advanced and innovative technologies in this world : Cryo-Electron Microscopy, which won the Chemistry Nobel Price in 2017. This was super amazing since it allows the researcher take real-time pictures of proteins that can’t be formed into large crystals!

Best book you’ve ever read?

Wuthering Heights

What’s the coolest thing you’ve experienced in the city you’re interning in?

I tried surfing in Half Moon Bay once. It was super cool and interesting with just a minor problem — I could barely stand up for more than three seconds.

What’s your most memorable Groupon moment or story so far?

My first day at Groupon is memorable to me. This is my first internship in my life and from the very first moment at Groupon I started working on a bunch of really cool projects!

What do your parents think you do?

My parents think I am developing “some programs” that keep sending the ads to customers…….

Shankar Golla – Database Administrator in Bangalore

Shankar Golla – Database Administrator in Bangalore

SHANKAR GOLLA

Database Administrator, EDW-DBA
Bangalore, India
Joined: 2016

Talking Teradata with Shankar, a Database Administrator in Bangalore.

What was your first role here? How did you get to where you are now?

I first started at Groupon as a Level 2 Teradata Database Administrator in Bangalore, India to support 24/7 database operations. I worked with cross-functional teams like DevOps and Development teams during Teradata upgrades and installations. As part of the Teradata downsizing project, I played a key role in identifying resource-consuming queries and worked with users on optimization to improve efficiency. I also identified stale scheduled jobs in our optimus tool and worked with the user to disable the job to reclaim CPU and IO resources. I identified users who ran frequent queries on a daily basis and asked them to reduce the frequency based on usage and core tables refresh.

I also managed our bi-weekly calls with the Teradata team to drive the Teradata open incidents and other issues pertaining to the system. Throughout my time, I have conducted multiple trainings and brown bag sessions for the user community to help understand the Teradata architecture and SQL optimization and best practices to write efficient and optimal queries. All of these challenging activities helped me to get where I am now: Level 3 Teradata Database Administrator.

Tell us about your job! What do you do exactly? What do you love about it?

Being a Database administrator, a typical day starts with addressing users’ questions and concerns raised through JIRA tickets. We work closely with the user community to optimize the Teradata SQLs. Currently, we have two Teradata systems: Production and Disaster recovery, data stored in Teradata used for analytics and decision making, it is our responsibility to make sure the systems are available for the business 24/7.

As part of infrastructure maintenance, we take care of software installation and maintenance, database backup and recovery, security, authentication, capacity planning, performance monitoring, database tuning, and troubleshooting. It is and was quite challenging to work with user community to understand their business requirements and to provide best practices and techniques to write efficient queries. I love to work with cross-functional teams during software upgrades and installations as we prepare the virtual machines for testing which gives me exposure to learn other database technologies like big data.

What’s the biggest challenge you’ve worked on?

Teradata cluster downsize. This was the first kind of project to reduce the cluster size and it had not been done in any of the customer locations. The system is used for daily ETL with strict SLA timelines and day-to-day analytics and critical reports with a user base of 700+ active users per month. To make it successful, we focused on every aspect of the system instead of just focusing on moving data from the existing system to the downsized system.

In order to do this, we had to do the following: reduce existing usage of system resources; finalize the nodes to be downsized based on the current capacity utilisation; determine how to reduce the cluster size with minimal or no impact to business; create backup and restoration strategies; and determine a rollback plan and strategy for critical datasets. Two phases of the project has to be completed within two months in conjunction with other priority high-priority projects. The project timelines are very challenging as we have to complete both the phases within the two months timeframe to minimize the rental cost.

What was your first impression of Groupon?

That it was a flexible environment and open place for all; I felt that I could reach out any of the engineers to get my job done. I also feel that Groupon provides many opportunities to learn new technologies.

What do your parents think you do?

They think I sit in front of a system in an air conditioned work environment.

Have you won any awards during your time here?

I received Engineering Excellence Award in Q2 of 2017.

Where can we find you outside work?

I like playing and watching cricket. I have two adorable kids with whom I like to spend most of my spare time.

Raj Shanmugasundaram – Database Administrator EDW in Dublin

Raj Shanmugasundaram – Database Administrator EDW in Dublin

RAJ SHANMUGASUNDARAM

Database Administrator – EDW
Dublin, Ireland
Joined: 2012

Data is integral to Groupon’s success and Raj’s work ensures that everything is running smoothly in the system, which he likens to being a traffic cop.

What was your first role here? How did you get to where you are now?

I first started at Groupon as an EDW employee in Chennai, India as a Database Administrator. I worked on various tasks related to Teradata Administration while helping grow the Chennai EDW team by interviewing candidates. During this time, I learned new technologies such as Postgres Administration and Alation Administration. The EDW team planned to spread across others locations to  support the business more widely, which is when I got the opportunity to move to Dublin, Ireland. Even in Dublin, I am the first EDW employee.

Tell us about your job! What do you do exactly? What do you love about it?

I am Teradata Database Administrator. Teradata is a fully scalable relational database management system. It is widely used to manage large data warehousing operations and acts as a single data store. The data that stores in Teradata used to take many business-related decisions. It is our responsibility to make sure the system is available for the business 24/7, controlling the user sessions and production pipeline sessions without impacting each other.

It was quite challenging to balance the user load and ETL pipeline loads. It’s kind of like allowing food trucks and consumer cars on the road at the same time. So we are basically the traffic cops, allowing everyone to move freely without impacting others. Aside from managing the traffic, we need to review the object definitions, data securities, and access controls. Automating the process is my favorite part. I’ve worked on many automation scripts like resource controller, notifications, etc.

What’s the biggest challenge you’ve worked on?

The Teradata Refresh Program. This was one of the biggest challenges and the complicated project may be we are the first one who did this activity. Reducing the active Teradata server is quite challenging. It involves many teams such as the network team, Teradata vendors, and the entire EDW team. We started our groundwork a long time ago and determined which options would have the least business impact while still being cost-effective. It was one of the great milestones for the EDW team.

What was your first impression of Groupon?

I felt it is an open place where I am allowed to do my job without any boundaries. Groupon allows me to grow as I wish. Very few places have this kind of opportunity.

How does your work connect to Groupon’s mission of being the daily habit in local commerce?

We are dealing with a large amount of data, which many teams use to make important business decisions, such as Marketing, Email, Finance, and Accounting. Most of Groupon’s core business teams rely on data.

Smart people, interesting work.

What’s unique about Groupon’s Engineering culture?

There are no boundaries to implement our ideas and there is a fantastic culture. Recently, during Teradata Refresh program (Teradata Downsize), we had a mammoth challenge: we had to do it in an efficient way with less downtime. We came up with a strategy of using the loaner system approach and it was a huge success and we are one of the Teradata customers who did this project. Aside from work, Groupon provides ample opportunities to get involved socially with programs such as the Volunteer-a-Thon, Parents@Groupon, Coder Dojo, and Tech Chit Chats. These activities help to build relationships with other teams while making the workplace very special.

We see that you are a member of Groupon Volunteers. Why did you join?

It is kind of like marketing, as it’s a way to tell the world about Groupon. And it is marketing with a good cause! I am also a member of both PRIDE@Groupon and Parents@Groupon.

Where can we find you outside work?

If I am in Chennai you can find me at the movie theatre during weekends, which is one of my favorite things to do. I am also interested in any sports activities like Staff Reply and Run in the Dark.

What do your parents think you do?

“The sun needs to learn from me how to work without taking rest.” My parents think that even I am working during my holidays. But it is my habit to check what is happening in Groupon.  

CTO Colin Bodell’s first of many visits to Groupon Bangalore Dev Centre

CTO Colin Bodell’s first of many visits to Groupon Bangalore Dev Centre

CTO Colin Bodell’s first of many visits to Bangalore Dev Centre

ENGINEERING

CTO Colin Bodell’s first visit to Groupon Bangalore Dev Centre

Colin Bodell’s first of many visits to Groupon’s Bangalore Dev centre instilled great confidence in the talented engineers working far across the globe toward Groupon’s long- and short-term strategies. On the first day of his visit, he addressed all of the engineering teams present in Bangalore: Platform and Data, Goods, Marketing, Customer Support Engineering, and Data Science. As part of his presentation, he introduced himself, his vision, strategy, and philosophy.

The key takeaways were:

  • People and relationships: his investment in people and building relationships in order to build a great team
  • Execution through ownership: his belief in getting cutting-edge execution done through ownership
  • Diversity: why and how he has been a leading proponent of diversity through forums such as Grace Hopper
  • Engineering links back to Groupon’s strategy: seeing how Bangalore engineers’ daily work can tie back to Groupon’s multi-year strategy
  • Communication as a key to success: he talked about his communication tenets

Over the next few days, he spent time diving deep into the major programs run out of Bangalore and meeting one on one with engineers. He listened intently and asked questions to the engineering teams and individuals in order to understand their achievements, aspirations, motivations, and explain his strategy in return. Engineers were super motivated by Colin’s energy and by having the accessibility to an SVP-level leader for the first time in Groupon Bangalore. 

It was not all work though. He also attended a potluck along with the engineering teams, met with all the diversity engineers of WiTB, and offered to extend any sponsorship/support to the group. On the last evening, we had a chance to see more of his fun side during an evening poolside dinner where he shared anecdotes from his past experiences, his journey, etc.

Darren Shaw – Data Scientist, Mobile Marketing in Seattle

Darren Shaw – Data Scientist, Mobile Marketing in Seattle

DARREN SHAW

Data Scientist, Mobile Marketing
Seattle, WA
Joined: 2017

Darren is out to change your stereotyped notion of data scientists. Sweater vests and pocket protectors? Not unless it’s about having fun at the office. Cutting-edge, mobile-first technology? Now we’re talking…

Tell us about your job! What do you do exactly? What do you love about it?

I have a couple main projects I’m focused on at the moment. The first is a massive experimental design that will observe how a variety of factors within our push notification framework affect our customers. This one is great because it combines the complex design of experiments and analytics with our number one priority: providing the best customer experience. The second project area I have is the application of machine learning to our push notification subscribers. Again, same outcome priority, but this time it is through implementing statistical modeling to historical trends within our customer base.

So what did you do before Groupon? How did you get here?

Prior to Groupon, I worked in advanced analytics at AT&T, applying machine learning to several of their email marketing campaigns.

You were at AT&T prior to Groupon, so can you talk about how you ended up here?  

I entered into AT&T through their new-grad tech program. This was a great opportunity to get acclimated to a large professional work environment and give thought to what I wanted out of my career. Though I was working in a data science group, I found a yearning to be closer to cutting-edge technologies and making a visible impact on customers.

Did you want to work at Groupon or did a recruiter reach out to you?

I pursued Groupon for a variety of reasons, some of which were the size and mission of the company. The size of Groupon meant they are a household name, but also lean enough to be nimble and quick-moving in the industry. Finally, their mission to be the daily habit in local commerce requires them to leverage the latest open-source tech as well as ensure that every day you are working on something that will impact the end-customer.

Once you were interviewing, was there anything that surprised you about Groupon that you didn’t know before?

I cannot understate the level of transparency I have observed during both my interview process and while working in the company. Transparency is an overused buzzword right now in nearly all organizations. It is one thing to say it and another completely to practice it.

How would you compare the culture/work at AT&T to Groupon?

When you fall into a 200,000+ person company you’ll be hard-pressed to find someone that is honest and clear regarding your performance, career ambitions, or how you fit into the current/future goals of the company. From the very first interview I had with Groupon, they gave me feedback – when asked 🙂 – and they continue to make sure the projects I work on help me build the skills to be where I want to be in one, three, and five years down the line.

What was your first role at Groupon?

I came in as a Data Scientist on the Push Notifications team. Luckily, I’m still on the same team, but our responsibilities have expanded slightly to be more mobile-focused.

How does your work connect to Groupon’s mission of being the daily habit in local commerce?

My work focuses on providing the best customer experience to our push notification customers. This mobile-only notification within a mobile-most company is a key opportunity to notify users of local selection while best serving their interests.

What’s your favorite local business?

Columbia City Bakery has a pretty great pistachio cake.

Tell us something that might surprise us about you.

Within a two week span last year, I went from circling the perimeter of Iceland in a camping van, to returning stateside, selling everything that didn’t fit in my car and driving 30+ hours to Seattle. It has been a great year!

What’s your favorite Groupon memory?

One morning the team surprised our manager by doing our best impression of him (aka all wearing sweater vests). Unfortunately, he didn’t come dressed for the occasion, but if I recall correctly he went out during lunch and picked up a vest so he didn’t feel left out.

What’s unique about Groupon’s Engineering culture?

One thing that has stood out to me from the first phone call I had with Groupon and all the way to all my current coworkers, is steadfast transparency. It is invigorating to work in an environment that values clear, honest communication at all levels.

Can you describe what makes Data Science at Groupon compelling or unique?

The world is your oyster with data science at Groupon. We are a mobile-first company, so you’re operating in a complex landscape. Projects range from recommender systems, to geolocation, to churn modeling, to the design of experiments, and much more. From soup to nuts, you’ll have ownership starting at technical development and up through presenting findings and recommendations to executive stakeholders. From my short time, I’ve been able to design large-scale experiments, complete modeling projects on our push-notification subscribers, and present my work to the highest levels of the company.

What was your first impression of Groupon?

Great meeting room names, even better view of the Puget Sound.

What’s the biggest challenge you’ve worked on?

I’m lucky enough that my current work is the most rewarding and challenging I have experienced. Last reported we have over 30 million Groupon customers and even more users active on our mobile platform. We are a mobile-most company. The ability to be a data scientist on such a large customer base in the fast-moving world of mobile marketing means I feel like I’m making a noticeable impact in the lives of our customers.

Where can we find you outside work?

Working to enjoy all that Seattle has to offer: hiking, camping, skiing, and even ferry-riding. Other than the outdoors, I enjoy playing guitar and am learning the piano.

What do your parents think you do?

My mom thinks I’m a Software Engineer on push notifications and is a big fan of the messages we send out every day. She has been subscribed since I started. My dad isn’t subscribed, but he has taken a liking to frequently checking out our Goods section and letting me know the great deals he finds. Recently, he purchased a new OBD-II car scanner from Goods. They both have their own unique way of supporting the work.

Tabs or spaces?

Spaces

Groupon is a massive data-driven experiment — this team helps run it

Groupon is a massive data-driven experiment — this team helps run it

Groupon is a massive data-driven experiment — this team helps run it

Groupon has tweaked and tested every corner of its e-commerce platform to find out precisely what makes customers click. Its platform is one of the world’s most optimized online destinations, but Groupon is still running daily experiments to add new features that increase business — and get rid of features that don’t.

Comprised of both data scientists and engineers, the Optimize team has built a tech platform for running those experiments with scientific rigor. We spoke with three team members about their efforts to reshape how Groupon thinks about data.

OPTIMIZE AT A GLANCE

TEAM DISCIPLINES: Engineers and data scientists.

WHAT THEY DO: Build tools that help Groupon iterate and improve on its customer experience; promote data literacy across the organization.

WHAT THEY DID BEFORE: The team includes statisticians, an astrophysicist, a music major, an economist and a theological studies major.

DID I READ THAT RIGHT?: Yep.

DEEPLY ROOTED: Optimize was one of Groupon’s founding teams.

INSPIRED BY: Pharma research, which pioneered a scientific way to “peek” at data and end experiments early.

THE STACK: Ruby on Rails, Node.js, Ember.js, Python, R, Hadoop/Hive.

What does Groupon’s Optimize team do?

Kristi Angel, data scientist: Experimentation is essential to product development at Groupon, and Optimize owns the platform for experimentation. Our software is used by product managers in developing new features and improvements on web and mobile platforms. We automate A/B testing, ensuring statistical rigor and provide reporting on experiment results.

I research statistical methodology to implement as features in our software, partner with product managers and leadership to develop a culture of statistical literacy, assist with the interpretation of experiment results and work with product analysts to design sound strategies for experimentation that does not fit within our framework. I also work on special projects related to the analysis of anomalous results and validation of data quality in our pipeline, for example.

Robb Broome, senior software engineer: The driving force is finding out whether the changes we make to our website are benefiting the business or hurting the business. As time goes by, we get better at finding solutions and ways of measuring impact that are more reliable and consistent.

What technologies power your experimentation platform?

David Oliver, engineering and product manager: Groupon’s website is built on a series of Node.js microservices, and our code is embedded as modules in those apps.

So, one of those Node modules that we own is Finch.js, which does two essential things: it buckets users into control or treatment groups, and it sends a message that the user saw the experiment. That message flows through a REST API, into Kafka and finally into Hadoop. We also have two Ruby on Rails applications. One is focused on consuming that data from Hadoop and piecing out the data in such a way that we can easily read it, and the other doubles as experiment configuration and the stats engine that powers our platform. Finally, we have an Ember.js app that our stakeholders use to create and manage experiments, and display experiment results.

If an experiment goes poorly, that can have a real impact on your bottom line. How do you balance those concerns against the need for statistical rigor?

Oliver: We use a technique called group sequential analysis, which was pioneered doing heart valve studies in the 1970s.

Angel: Imagine a clinical trial in which an experimental treatment is actually harming people. Ethically, one would want to stop the trial immediately, but statistically it is a bad thing to watch your data as it accumulates. All measurements have a natural variation and this randomness can easily be mistaken for a signal. We really only want to analyze the data once we know we have enough observations to be confident that what we are seeing is signal and not noise.

Group sequential analysis is a way to “peek” at the data in a controlled way, periodically checking in on the experiment in a statistically rigorous manner that limits false positive results. It turns out, a life-saving mechanism in pharma research is a revenue-saving mechanism at Groupon. We can more quickly capitalize on features that increase revenue and limit exposure to features that lose revenue.

What are the most interesting technological problems your team is solving?

Angel: We are working on arriving at the optimal attribution models across different areas of our business. For example, the attribution of a purchase to a specific experience — a home page feature, an email, a push notification — likely has a different window of time where we can reasonably say a specific purchase is a result of a specific experience.

Features that reduce friction on a site are likely to have a more immediate effect, whereas an email campaign might have an impact over a longer time frame. Mathematically, how do we find that optimal window? Technically, how do we implement and support a number of attribution methods for our framework?

Smart people doing interesting work

Groupon has a long history of experimentation, and a lot of data to work with. Does that present unique challenges?

Broome: Groupon is already highly optimized, which means you need better science and better math. In the olden days, it was easy to make huge changes that you didn’t need very sophisticated systems to see. We also have to pay very close attention to whether the data is getting in on time, because there are billions of transactions coming in, and the paths are pretty complicated.

What is one important change to how Groupon works that has emerged from the Optimize team’s work?

Angel: Finch Express, which is what we call our platform, has changed the way Groupon does A/B experimentation. Before its existence, experimentation was a pretty loose concept. Experimenters would monitor results day over day and run things until they looked “good” — usually meaning that the data was susceptible to the experimenter’s biases.

Today, we do not display data about the metric of interest until the experiment’s conclusion. That prevents experimenters from checking the results before enough data has accumulated and limits false conclusions.

How does that impact Groupon’s approach to data?

Angel: Our platform further moves the needle by shifting our organizational culture. It isn’t hard nor is it enough to be data driven. We must also be data literate. By building out a sophisticated platform and committing to comprehensive support for our consumers, we grow data literacy in the organization and as a result we’re able to make better decisions.

What drew you to Groupon?

Oliver: The Optimize team and getting to work both with programming languages I was already familiar with, like Ruby and JavaScript, as well as getting to touch other languages like Scala and Clojure. Our team doesn’t use Scala or Clojure anymore, but I love that Groupon as a whole isn’t afraid to try different languages and technologies — and just as easily move on if they’re not working. I was also attracted to having lots of moving parts and getting to work with all the different teams in the company.

What’s your favorite thing about Groupon’s culture?

Angel: I always felt supported in my personal development. Leadership here has a strong interest in making sure that you’re doing the kind of work you want. And although people are really smart, they’re also really fun. There’s a pervasive sense of humor across the company that I really appreciate, because I’m kind of a goofy person.

What does your team look for in developers?

Angel: We obviously look for technical skills; coding ability matters. Our team works in Ruby and Javascript and is full stack. But a great candidate might actually be a Java developer or have limited experience with front end development. We’re also looking for someone who is curious and humble, with good analytical intuition, and who has a strong sense of ownership, good communication skills and a commitment to excellence.

Oliver: It’s important to us that people are good communicators and who can explain complex concepts in a clear and straightforward way. We also look for professionalism, which is one of those qualities that’s hard to describe, but you know it when you see it. You have to be able to represent the team to the rest of the company.

Groupon is a massive data-driven experiment — this team helps run it

Groupon is a massive data-driven experiment — this team helps run it

Groupon is a massive data-driven experiment — this team helps run it

via BuiltIn Chicago

Groupon has tweaked and tested every corner of its e-commerce platform to find out precisely what makes customers click. Its platform is one of the world’s most optimized online destinations, but Groupon is still running daily experiments to add new features that increase business — and get rid of features that don’t.

Comprised of both data scientists and engineers, the Optimize team has built a tech platform for running those experiments with scientific rigor. We spoke with three team members about their efforts to reshape how Groupon thinks about data.

OPTIMIZE AT A GLANCE

TEAM DISCIPLINES: Engineers and data scientists.

WHAT THEY DO: Build tools that help Groupon iterate and improve on its customer experience; promote data literacy across the organization.

WHAT THEY DID BEFORE: The team includes statisticians, an astrophysicist, a music major, an economist and a theological studies major.

DID I READ THAT RIGHT?: Yep.

DEEPLY ROOTED: Optimize was one of Groupon’s founding teams.

INSPIRED BY: Pharma research, which pioneered a scientific way to “peek” at data and end experiments early.

THE STACK: Ruby on Rails, Node.js, Ember.js, Python, R, Hadoop/Hive.

What does Groupon’s Optimize team do?

Kristi Angel, data scientist: Experimentation is essential to product development at Groupon, and Optimize owns the platform for experimentation. Our software is used by product managers in developing new features and improvements on web and mobile platforms. We automate A/B testing, ensuring statistical rigor and provide reporting on experiment results.

I research statistical methodology to implement as features in our software, partner with product managers and leadership to develop a culture of statistical literacy, assist with the interpretation of experiment results and work with product analysts to design sound strategies for experimentation that does not fit within our framework. I also work on special projects related to the analysis of anomalous results and validation of data quality in our pipeline, for example.

Robb Broome, senior software engineer: The driving force is finding out whether the changes we make to our website are benefiting the business or hurting the business. As time goes by, we get better at finding solutions and ways of measuring impact that are more reliable and consistent.

What technologies power your experimentation platform?

David Oliver, engineering and product manager: Groupon’s website is built on a series of Node.js microservices, and our code is embedded as modules in those apps.

So, one of those Node modules that we own is Finch.js, which does two essential things: it buckets users into control or treatment groups, and it sends a message that the user saw the experiment. That message flows through a REST API, into Kafka and finally into Hadoop. We also have two Ruby on Rails applications. One is focused on consuming that data from Hadoop and piecing out the data in such a way that we can easily read it, and the other doubles as experiment configuration and the stats engine that powers our platform. Finally, we have an Ember.js app that our stakeholders use to create and manage experiments, and display experiment results.

If an experiment goes poorly, that can have a real impact on your bottom line. How do you balance those concerns against the need for statistical rigor?

Oliver: We use a technique called group sequential analysis, which was pioneered doing heart valve studies in the 1970s.

Angel: Imagine a clinical trial in which an experimental treatment is actually harming people. Ethically, one would want to stop the trial immediately, but statistically it is a bad thing to watch your data as it accumulates. All measurements have a natural variation and this randomness can easily be mistaken for a signal. We really only want to analyze the data once we know we have enough observations to be confident that what we are seeing is signal and not noise.

Group sequential analysis is a way to “peek” at the data in a controlled way, periodically checking in on the experiment in a statistically rigorous manner that limits false positive results. It turns out, a life-saving mechanism in pharma research is a revenue-saving mechanism at Groupon. We can more quickly capitalize on features that increase revenue and limit exposure to features that lose revenue.

What are the most interesting technological problems your team is solving?

Angel: We are working on arriving at the optimal attribution models across different areas of our business. For example, the attribution of a purchase to a specific experience — a home page feature, an email, a push notification — likely has a different window of time where we can reasonably say a specific purchase is a result of a specific experience.

Features that reduce friction on a site are likely to have a more immediate effect, whereas an email campaign might have an impact over a longer time frame. Mathematically, how do we find that optimal window? Technically, how do we implement and support a number of attribution methods for our framework?

Smart people doing interesting work

Groupon has a long history of experimentation, and a lot of data to work with. Does that present unique challenges?

Broome: Groupon is already highly optimized, which means you need better science and better math. In the olden days, it was easy to make huge changes that you didn’t need very sophisticated systems to see. We also have to pay very close attention to whether the data is getting in on time, because there are billions of transactions coming in, and the paths are pretty complicated.

What is one important change to how Groupon works that has emerged from the Optimize team’s work?

Angel: Finch Express, which is what we call our platform, has changed the way Groupon does A/B experimentation. Before its existence, experimentation was a pretty loose concept. Experimenters would monitor results day over day and run things until they looked “good” — usually meaning that the data was susceptible to the experimenter’s biases.

Today, we do not display data about the metric of interest until the experiment’s conclusion. That prevents experimenters from checking the results before enough data has accumulated and limits false conclusions.

How does that impact Groupon’s approach to data?

Angel: Our platform further moves the needle by shifting our organizational culture. It isn’t hard nor is it enough to be data driven. We must also be data literate. By building out a sophisticated platform and committing to comprehensive support for our consumers, we grow data literacy in the organization and as a result we’re able to make better decisions.

What drew you to Groupon?

Oliver: The Optimize team and getting to work both with programming languages I was already familiar with, like Ruby and JavaScript, as well as getting to touch other languages like Scala and Clojure. Our team doesn’t use Scala or Clojure anymore, but I love that Groupon as a whole isn’t afraid to try different languages and technologies — and just as easily move on if they’re not working. I was also attracted to having lots of moving parts and getting to work with all the different teams in the company.

What’s your favorite thing about Groupon’s culture?

Angel: I always felt supported in my personal development. Leadership here has a strong interest in making sure that you’re doing the kind of work you want. And although people are really smart, they’re also really fun. There’s a pervasive sense of humor across the company that I really appreciate, because I’m kind of a goofy person.

What does your team look for in developers?

Angel: We obviously look for technical skills; coding ability matters. Our team works in Ruby and Javascript and is full stack. But a great candidate might actually be a Java developer or have limited experience with front end development. We’re also looking for someone who is curious and humble, with good analytical intuition, and who has a strong sense of ownership, good communication skills and a commitment to excellence.

Oliver: It’s important to us that people are good communicators and who can explain complex concepts in a clear and straightforward way. We also look for professionalism, which is one of those qualities that’s hard to describe, but you know it when you see it. You have to be able to represent the team to the rest of the company.

Khushbu Agrawal – Data Warehouse Engineer in Bangalore

Khushbu Agrawal – Data Warehouse Engineer in Bangalore

KHUSHBU AGRAWAL

Data Warehouse Engineer
Bangalore, India
Joined: 2017

So tell us about what it’s like being a Data Warehouse Engineer at Groupon.

I create something meaningful for everybody. Yes, there is a lot of data generated everywhere. As a Data Warehouse Engineer, I convert data into meaningful information which is critical for the business to make more informed decisions in order to grow the company.

What does a typical day look like for you?

My day basically in office starts up with the standup call, which gives the team a good head start. I then work on a couple of user requests along with small breaks in between. Yes, I don’t forget enjoying time with my colleagues in the chaos of work, too. At the end of the day, it gives me a sense of accomplishment with work, fun, and food

What’s the most challenging thing you’ve worked on so far?

GDPR compliance is keeping everyone on their toes. Lots of old technology is going off and users sensitive information is to be kept hidden. I am working on this piece of work which is quite challenging.

What do your parents think you do?

Who knows, ha! They think I do stuff on a laptop.

What has been surprising about Groupon so far?

Groupon has every type of engineer. Any technology, you name it, Groupon has a skilled resource for it. Great learning with a great future is guaranteed here.

What’s your favorite programming language?

I love coding in Unix.

What’s unique about Groupon’s Engineering culture?

It’s open and full of unique, talented people.

What’s your favorite conference room?

What’s your favorite Groupon memory?

Participating in GEEKon, our global internal hackathon. I had just joined Groupon two months prior and I was so thrilled to see people excited and geared up for the event. Each team came up with unique ideas, and I too took part in it. It was fun working with and learning from other teams. My team worked on creating a multi-dimensional analysis (OLAP) on Hadoop using open source Apache Kylin. This provided us low latency and higher performance even in the TB of data. I was the Architect and designer of this project. This project gave us good learning and exposure!

What are your hobbies outside of work?

Traveling, hanging around with friends, and last but not least, online shopping. 🙂

\

Kristi Angel, Data Scientist

Kristi Angel, Data Scientist

KRISTI ANGEL
Data Scientist
Chicago, IL
Joined: 2015 & 2016
“That’s great, it starts with an earthquake…Birds and snakes, and aeroplanes…And Kristi Angel is not afraid…”

What was your first role here? How did you get to where you are today?

I started as a Contract Tableau Analyst while in graduate school. I left that position to focus on finishing my M.S. in Statistics. I had fallen in love with the culture at Groupon. Having made some great connections during my brief tenure, I was able to find the perfect role for myself doing challenging work on a super smart and supportive team with incredible leadership.

After I graduated, I returned as an Experimentation Analyst. The needs of my team shifted and I grew into taking on more complex projects. Here at Groupon, I’ve been given a lot of autonomy in developing my career and identifying projects that are right for our platform but also aligned with my own professional development goals. After eight months, I spoke with my manager about a change in title to Data Scientist to more closely reflect the work that I was doing. I was humbled by the support I received from my peers and leadership across Groupon to make that happen!

So tell us more about your job! What do you do every day? What do you love about it?

I am a Data Scientist working in the Experimentation space. We are a team of Developers and Data Scientists and together we build Groupon’s A/B Experimentation platform.

In my role, I work with Product Managers and Engineers to help develop sound experimentation design, to interpret results, and to debug experiment configuration (and occasionally, our platform 😉 ). I also research methodology for new features on our platform and help develop and validate our data pipeline and import process. Additionally, I help guide the roadmap for our team, partner with other organizations within Groupon to grow relationships and improve operational efficiency, and advocate for sound statistical practices and data literacy for all!

I love that I get to stretch my leadership skills, make mistakes, and learn from my peers. Above all, I love that every now and then, a great idea strikes and I get to make someone else’s job a little easier or I find a novel way to increase the value my team delivers to Groupon. I owe this to my coworkers who have trusted my perspective and skill set, elevating my voice from Day 1.

What’s your favorite part about your team’s Software Development process?

We are very democratic and thoughtful about process. Also, we have a lot of say in the features we work on. We work closely with our customers and are best positioned to assess the impact a new feature can bring. If anyone on our team has an idea about a feature or improvement they would like to see implemented on our platform and they are willing to really advocate for it and prove its value, there’s a good chance s/he will be able to see it through to production.

What did you do before Groupon?

Blood, sweat, tears, and a lot of eraser dust! (That was a math joke.) Fun Fact: I was a waitress for 20 years before I joined Groupon! I worked while studying Physics as an undergrad and then while completing my M.S. in Statistics.

Ready to make an impact?

You’re involved with PRIDE@Groupon, Women@Groupon, and Groupon Volunteers. Why do you participate? 

I joined because I care. And there’s so much power in joining forces with others who care! Interestingly, one of the unintended side effects of joining Groupon’s Employee Resource Groups is that I have made some of the strongest connections through participating in ERG events. The camaraderie that develops through participation means you always have your own personal cheerleading section. And everybody needs one of those from time to time!

What’s unique about Groupon’s Engineering culture?

Everyone here is incredibly kind, helpful, empathetic, super smart, and fun! I’ve learned so much about being a person in the past two years and I owe that to my peers, my mentors, and strong leadership all around.

What’s something that would surprise people to learn about Groupon Engineering?

Our people genuinely care about people. This is a place where we encourage an experimentation mindset, failing fast and moving forward. We really value the process of giving and receiving feedback at all levels. It’s how we grow.

Name your favorite programming language.

I’m a statistician by trade so, R.

How does your work connect to Groupon’s mission of building the daily habit in local commerce?

We provide tools for Product to rapidly and iteratively make improvements to our mobile apps and our website. Building tools to help guide efficient and effective product development helps Groupon to capitalize on optimizations that drive the daily habit.

Tell us about the most challenging thing you’ve worked on here.

I just rewrote our experimentation platform’s stats libraries in R. The goal was to create experiment metadata for a project I am working on. It’s a rather complex system with nuance and so validating my model with our platform has been trying at times. But, I was able to identify a few places in our platform where we could improve our methodology. And the metadata that I have created will serve our Data Science team on a number of future projects.

What’s your favorite Groupon memory?

This past September, Groupon participated as a sponsor at Chicago Women In Technology where our VP of Engineering Operations, Erica Geil, spoke. Later that same beautiful late summer day, the Women@Groupon Employee Resource Group hosted a happy hour on our roof top deck where Silvija Martincevic, GM of Health, Beauty, and Wellness, spoke about her own career path. It was just an incredible day, taking in the sunshine with strong, successful, empowered women and learning from everyone’s unique journey.

What’s your favorite local business?

Can I have three? (Yes, you can!)

Lombard Skate Shop: Steve is super knowledgeable about getting a person into the right skates and gear for them.

Play: A children’s toy store in Logan Square with really unique and smart gifts for kids of all ages.

And Wasabi: It has been a pleasure to watch my favorite ramen shop grow over the years. I used to be one of a few in there weekly. Now there is a line out the door nightly.

What do your parents think you do?

my_family <- ¯\_(ツ)_/¯

What keeps you fulfilled outside of work?

I can generally be found cooking, eating ice cream, checking out new restaurants, roller skating, chilling with my cat-dudes, traveling, or seeing live music.

Tell us something about you that might surprise us!

What about me would surprise people? That I love The Muppets? I was in the Army for 6 weeks? I attended 9 different colleges and universities? I was an undergrad for 14 years? I was a waitress for 20 years? I know all the words to “It’s the End of the World As We Know It (And I Feel Fine)”? I’m just throwing things at the wall here.

Data Driven Chicago (Second Edition)

Data Driven Chicago (Second Edition)

Data Driven Chicago

Ilhan Kolko (Echo)
Andrew Lisy and Laura Hamilton (Groupon)
Tyler Hanson (Reverb)
Mary Feigenbutz and Greg Reda (Sprout Social)
Laurie Skelly and Elizabeth Cleveland (Trunk Club)
Moderated by Alli Diedrick (Built In Chicago)
November 2, 2017

Showcasing some of the great data-driven and machine-learning talent here in Chicago. Brought to your by Groupon, hosted by Built In Chicago.

See more videos from Groupon and our friends.

Groupon is a massive data-driven experiment — this team helps run it

Groupon is a massive data-driven experiment — this team helps run it

DATA SCIENCE & ENGINEERING

29 August 2017

Groupon is a massive data-driven experiment — this team helps run it

Groupon has tweaked and tested every corner of its e-commerce platform to find out precisely what makes customers click. Its platform is one of the world’s most optimized online destinations, but Groupon is still running daily experiments to add new features that increase business — and get rid of features that don’t.

Comprised of both data scientists and engineers, the Optimize team has built a tech platform for running those experiments with scientific rigor. We spoke with three team members about their efforts to reshape how Groupon thinks about data.

Read the complete article at BuiltInChicago

How the Flux team at Groupon uses insights from the past to predict the future

How the Flux team at Groupon uses insights from the past to predict the future

DATA SCIENCE & ENGINEERING

22 July 2017

How the Flux team at Groupon uses insights from the past to predict the future

In case you were wondering, the Flux Team at Groupon is named after the 1985 classic, “Back to the Future,” because its members use insights from the past to predict the future. Fluent in both machine learning and big data, the team serves to translate for the company’s data scientists and engineers, who use vastly different coding languages and approaches. In so doing, Flux makes sure machine learning can be put into use very quickly, whereas in the past it could take months.

Recently, we spent time with the group to find out how they work across the approximately 2,000-person Chicago location (7,000 globally) to keep the “flux capacitor … fluxing.”

Read the complete article at BuiltInChicago