Why your product will fail without a compelling story crafted and told with data

Why your product will fail without a compelling story crafted and told with data

Why your product will fail without a compelling story crafted and told with data

Laura Hamilton
Group Product Manager
July 18, 2018

Thursday, the Chicago chapter of Women in Product hosted a panel about “The Power of Storytelling in Product Development.” In Thursday’s event, three female product leaders shared with us their stories about how compelling storytelling leads to successful products — and how a weak or missing story will cause a product to flounder or fail.

The event was sponsored by Uptake, a predictive analytics SaaS provider focused on the industrial space. The panel discussion was moderated by Taylor Klassman, Senior User Experience Researcher at Uptake who leads research in the Renewable Energy vertical. In addition to the usual client interviews and meetings with PMs, Taylor “goes the extra 300 feet” (vertically) by climbing wind turbines to watch end users in their daily workflows.

The panel was composed of O’Dealya Price (Senior Product Manager at Uptake on the IoT Edge Team), Kelly Bishop (VP Product and Design at Fusion Media Group), and Shelby King (Senior Product Manager at Venmo).

Human beings have a fundamental need for stories, a need to craft narratives out of data. In fact the human need for narratives is so compelling, so innate, that people routinely craft narratives around random data, creating a storyline where there isn’t one. Parents have been telling stories for babies since the beginning of the human race, using stories as the primary method to teach children about the world around them.

The panelists emphasized the importance of being customer-obsessed, of crafting the product storyline based on data and customer research, and of evangelizing the product story with the product development team. “All of that rich context, all of that data that you have been looking at, all of those conversations that you have had with customers, the user research that you have done, the context you have gotten from leadership on what the business goals are for the next 6–12 months or the next 5 years. Everything you have in your head, get it out of your head. Share it with the people working on your project,” said King. She explained that at Venmo, product managers first write a Google doc that illustrates the customer problem, brings in data to contextualize the problem, then defines the solution and KPIs. Next, she shares the doc as broadly as possible throughout the organization — with engineering, with risk, with legal, with marketing, and with PR.

King shared an example about how she used the customer’s voice to tell the product story to her engineering team to tell the story of the Venmo Debit Card, launched last week. She told us how she compiled a list of customer tweets indicating a customer need for the debit card product. Customers were tweeting @venmo, lamenting the fact that the ice cream truck on their street did not take Venmo. King explained to her engineering team how the Venmo card solved the customers’ problem and provided immediate ice cream paid for by their venmo accounts. “If you’re able to bring through examples of what the customer is saying, that can be really powerful,” King explained.

Smart people doing interesting work

Writing the product narrative can also help with the strategy. Bishop shared with us a story about an initiative that her team was ready to begin work on, but then when she took a step back to write the story concluded that there was no “there” there. “We stepped back to look at the story, and we sunset the project because it wasn’t a good use of the team. Always second guess some of that pressure,” said Bishop.

The product panelists emphasized the importance of knowing your audience, and tailoring your story appropriately. “Knowing your audience is a big part of being a successful story. What I do is try my best to understand what the people I’m telling the story to care about. Tailor your story; everybody doesn’t need to know everything,” said King. It’s the product manager’s job to gather all of the data, all of the customer research, all of the anecdotes and all of the information — and then to edit it appropriately for the given audience. “The most important phase of this,” explained Bishop, “is when you edit your story.”

And it’s important to watch the audience, to see if the product story is hitting home or is missing its mark, and to adjust on the fly. “When I explain things and there’s silence afterwards, that’s an indication I didn’t explain something, didn’t understand the audience,” said Price. But telling the story isn’t just a one-time event that happens at the project kick-off. It’s a continual effort, done through many meetings, emails, slack messages, and documents. A product leader needs to set the project goals and KPIs at the outset, then have regular check-ins to review progress and reiterate the story. These regular check-ins help keep the team energized and also helps to drive improvements and optimizations in the product and processes.

===

Laura Hamilton is a Group Product Manager at Groupon. We’re hiring Product Managers in multiple locations. Want to work with us? Browse our current job listings or learn more about us on our Product page.

How to design a well-designed offsite

How to design a well-designed offsite

How To Design a Well-designed Offsite

Helena SEO
Director, Product Design
June 28, 2018

UX Design at Groupon is an organizationally-centralized but geographically-distributed team. There’s one time of year when everyone in the team — designers, researchers and content strategists — come together for our annual spring offsite.

The offsite has provided the team great opportunities to learn new skills, expand creative thinking, and most importantly unify and strengthen the team. It’s been an annual tradition since 2013, and we feel very fortunate to be a part of the company who values and invests in team building.

Our team just had this year’s offsite for three nights in beautiful Monterey, CA, in early May. It had a successful turnout, scoring 4.86 out of 5 in overall satisfaction based on the internal post-event survey.

This article isn’t meant to be another chronological documentation of the event for the self-indulgent celebration. Rather, the goal is to share a few useful tips and ideas on how we were able to achieve a nearly perfect satisfaction score at the offsite with those of you who are also planning for a similar event in the future.

I find that designing an offsite somewhat similar to the UX design process. You need to understand main demographics first, empathize their pain points and interests, go through multiple rounds of diverging and converging, and finally deliver the most creative possible solution within constraints.

1. Get the logistical headache out of everyone’s way

Similarly to any personal trip, having to rent, drive or park a car can be one of the biggest logistical headaches during the team offsites. Specially when nearly 30 people are in motion all at once, there are likely more chances of unexpected unfortunate instances. Those instances can take fun and excitement away from the actual offsite event.

This year, we decided to avoid the issue by investing in a shuttle service: One shuttle taking the locals from the Palo Alto office and another one taking the travelers from San Jose airport. Nobody in the team had to drive, and we could get around downtown via Uber/Lyft easily once we were in Monterey.

2. Involve everyone in planning, but with a specific ownership assignment

The more ownership the team feels themselves, the more active engagement and participation there will be at the offsite. Instead of a top-down planning, we decided to take on a more autonomous approach this year and truly practice “team of the people, by the people, for the people.”

Each team member owned some part of planning and operated in a designated committee, whether it was about running a skill training, selecting a dinner venue, producing event schwag, coordinating transportation, etc. Slack was an effective tool for the specific committee discussions.

Because of the shared responsibility, no one was overly burdened by the planning task this year. Also, as there was genuine, shared ownership of the event, everyone sincerely cared for and contributed to the success of the offsite.

3. Leverage the internal talents to better everyone

You’d be amazed to know how much diverse talent and knowledge exist internally, and how great it is to learn from each other. These internal knowledge transfer can be sometimes more applicable and practical than learning from an expensive external speaker, because the training content is more contextual to the daily task at hand.

We recruited the trainers within the Design team for both technical and soft skills a couple of months in advance, and asked everyone in the team to sign up for the classes of their interest. The sufficient lead time provided the trainers enough time to prepare for the class materials, strategize the right interaction model with the trainees, and design the difficulty level properly.

In this year’s offsite, the technical trainings were all about prototyping in a tool of individual’s choice: Framer, After Effect and CSS. As our design team focuses more on innovation of micro interaction and effective collaboration with engineering team, enhancement of the prototyping skill becomes more critical across the board.

Soft-skill trainings had two breakout sessions: How to improve storytelling skill using user-centered case and how to understand nuances of working with data. Both sessions were designed to help the collaboration with the cross-functional teams and empower the design organization further.

These internal trainings enabled continuing Q&As and motivated each other to hone in skills further not only during the offsite but also the post-offsite.

4. Design one-of-a-kind activities through a creative spin

A few images may come to a mind when you picture a corporate offsite: A roundtable for introducing each person, an icebreaker game, a blindfold challenge, and if budget allows, perhaps a ropes course.

We didn’t do any of that in this year’s offsite. Instead, we considered how these activities can uniquely leverage our creativity as a cross-functional design team. Here are a few activity examples the team thoroughly enjoyed and therefore scored high in satisfaction:

A. Introduction of everyone via a trading card

Just for a background, each quarter, individuals in UX Design team are paired up with a “buddy” who they normally don’t work with. Buddies go through a few conversational activities to discover each other over the quarter, and then present each other in various forms at the end of the quarter. This activity has proven to an effective way to build a supporting system within the team and create a collaborative, friendship-based culture.

The outcome of this quarter’s buddy activity was a trading card. We used it as a way to introduce everyone in the team at the outset of the offsite. Everyone had such an interesting story to tell about his/her buddy creatively, and the session itself played a nice ice breaker as we were heading to the three-day eventful offsite.

(Credit for designing the buddy trading-card program goes to David Schnorr and Matt Hanson! Stay tuned for their upcoming article that’s dedicated to this particular program.)

B. Team building through building a boat — a real boat that floats!

We wanted to pick a unique team building activity that involves creative strategy and collaborative problem solving to reflect our team’s strength, and boat-making activity perfectly satisfied our goal. We worked with Adventure Associates to help facilitate this event.

We formed 6 teams, each with 4–5 team members. Every team was given art supplies and asked to design, engineer and construct a boat in 2 hours. After the construction, a sailing championship began and competition was on! Both teams of the fastest speed and the best aesthetics were awarded.

This activity truly attested to everyone’s creativity, craftsmanship, and collaboration. It was also amazing to see how quickly role delegation autonomously happened based on team member’s strength and skill set without an explicit instruction.

(Boat on the left: Winner of the speed category; Boat on the right: Winner of the aesthetics category)

C. Laptop-less design activity

Most of us spend enough hours on laptop everyday. So we tried to avoid activities that involved a laptop except for the technical skill training sessions.

Among many great activities, one of the most successful creative exercises was the block-art building led by our own Kevin Fox. Everyone was given Lego blocks and a grid template to build any design of their choice in 30 mins. Despite the extreme time pressure, the team was able to create an impressive collection of block arts. This was a great way to stretch our creative (and mathematical) muscle in a different way.

Smart people doing interesting work

5. Leave some room for personalization

In the group of nearly 30 people, there’s a broad spectrum of personalities, lifestyle choices, and skill levels. We designed the offsite with a structured framework with mostly maximizing togetherness, but with flexibility in some activities such as yoga in the morning and social mingling at night.

Skill trainings were done in a variety of breakout sessions where each person could get focused training based on their interest.

We also collected everyone’s dietary restrictions and menu preferences in advance, so that the hotel could accommodate most needs appropriately.

6. Start and end strong with meaningful celebration

Start: Our event was kicked off with State of Union where the team’s major achievements over the past year were acknowledged. The achievements included not only the success of the products but also the improvement of the process and operations. It was a great way to commemorate the countless accomplishments together and witness the team’s visible growth over the year. And the celebration fueled the positive energy to the rest of the offsite.

End: After the long three days of offsite, people usually get very tired, and all activities tend to become a big blur. To end the event strong with something to remember by, we played a video capturing all moments of the offsite. A photo/videography committee was formed in advance to capture the moments throughout the event. The below is the team video — hope you enjoy!

(Video created by Soffee Yang)

Closing Thoughts

We became a stronger and more collaborative team after the offsite. I hope our learnings and tips will also help many teams lead a satisfying offsite event. If you have any other ideas about the team event planning, please share them in the comments section so we can learn from each other!

I’d also like to take an opportunity to give a big shout out to the awesome Groupon Design team (A.K.A. Design Union) for everyone’s amazing ownership and participation at this event. Also special thanks to Pratik Mall, Ling Hu, Tracy Ulin, Kevin Mendoza and Tae Kim for helping me with this article.

===

Helena Seo is a Director of Consumer Experience Design at Groupon. We’re hiring all levels of designer/design leaders and content strategists. Want to work with us? Browse our current job listings or learn more about us at Groupon Design Union.

Have You Tried the “Sorkin Stand”?

Have You Tried the “Sorkin Stand”?

Image from Funny or Die, used without permission

Have You Tried the “Sorkin Stand”?

A Daily Scrum Method to Improve Your Team’s Focus and Make Your Employees Happier

Emily Wilson
Technical Program Manager
June 22, 2018

Working in an agile environment comes with one certainty: the Daily Scrum, also known as the stand-up meeting.Using this meeting to increase team or project efficiency while keeping the meeting itself efficient is often a tricky balance that many find themselves regularly iterating upon. Enter: the Sorkin Stand.

Aaron Sorkin is a highly respected screenwriter, well-known for his heavy use of the Walk and Talk, a storytelling technique that adds a heightened sense of drama or urgency to scenes by specifying that characters walk as they converse. While Sorkin’s The West Wing is fondly remembered for this technique, my personal favorite example is when Sorkin himself parodies his style in a scene of the 30 Rock episode “Plan B,” explaining screenwriting to Tina Fey’s character, Liz Lemon, while walking her in a circle.

Whenever department-wide meetings ran past time and cut into our scheduled Daily Scrum, my team would take care of “stand” (as we affectionately call it) as we walked back to our desks. We were already on our feet, so why not? Not only could we make up for lost time, but distractions were limited; my team and I could more easily think about the day as a whole without relying on our monitors in the moment. But why not start doing Scrum with movement every day, not just when it’s necessary?

Just as removing chairs from a Scrum has made it more efficient, I believe that adding movement is the next evolution of the Daily Scrum.

Smart people doing interesting work

Just as removing chairs from a Scrum has made it more efficient, I believe that adding movement is the next evolution of the Daily Scrum. Studies have shown the positive effects of moving while trying to learn and retain information and how regular exercise can improve thinking skills. Maybe it’s the next evolution of meetings altogether.

On an anecdotal note, my best friend and I would go to the gym to walk on the treadmills as we studied for college exams. We alternated between silent reading and quizzing each other. Introducing movement increased our engagement with the class, leading us to to discuss our studies any time we were at the gym together, not just before exams. And, while I am not a runner, my pacing was much better during a recent 5k where I opted to listen to an audiobook instead of music. Whether interaction makes exercise more productive or vice versa might be a “chicken or the egg” scenario, but what matters is that they both contribute to the health and happiness of your employees.

Every team is different with its own unique challenges. If your team is global, you may be reliant on a stagnant camera for Daily Scrums and team meetings. Accessibility should always be considered as well. But if your team is able to give the “Sorkin Stand” a try, please do. Let me know what you think! If you’ve brainstormed ways to make the Sorkin Stand work in a way that’s more inclusive, please share that too.

Data-Driven Product Management at Groupon

Data-Driven Product Management at Groupon

Data-Driven Product Management at Groupon

Laura Hamilton
Group Product Manager
May 22, 2018

Introduction

At Groupon, we have a very data-driven philosophy of product management. In this blog post, I’ll talk through how we approach product ownership in a data-driven way, from financial forecasting to roadmapping to feature development to experimentation.

Hippo

https://www.howgoogleworks.net/

Financial Forecasting and Roadmap Creation

For every candidate feature, we calculate the projected financial upside according to the following formula:

feature_revenue_forecast =
expected_lift x platform_factor x success_probability x platform_revenue

Where:

  • feature_revenue_forecast is what we are trying to calculate (the expected revenue from the feature)
  • expected_lift is the increase in conversions we expect from users in the treatment group vs. users in the control group. The vast majority of experiments fall between -1.0% and +1.0% lift.
  • platform_factor is what percent of all users of the platform (whether iOS, android, mobile web, or desktop web) are part of the experiment. For example, for a test on the checkout page on mobile web, the platform factor will be 100%—100% of users who place an order on mobile web visit the checkout page during their journey. For a test on the getaways deal page, the platform factor will be much smaller, so the overall financial impact of the test will be smaller.
  • success_probability is a haircut we apply to take into account that not all experiments will succeed. In fact, only about 30% of our experiments are successful. The success_probability for a given feature could be greater than or lower than 30%, depending on how confident we are that the experiment will succeed. For some experiments that are primarily for strategic reasons, such as maps improvements, we will use a success_probability of 90% or 100%.
  • platform_revenue is the total revenue generated by the platform. For example, the platform_revenue for iOS is the total revenue from orders placed via the iOS app.

With this formula, we have a consistent and data-driven way to estimate the upside from each proposed initiative. Then, once each experiment concludes, we compare our estimates to the actual results, and over time we refine our estimations.

We use these estimated upside figures to create product roadmaps. In order to prioritize initiatives and determine the cutlist, we need to introduce another data point—the engineering effort required. Then, we use the following formula to calculate the ROI of each feature:

ROI = feature_revenue_forecast / level_of_effort

We then stack rank features according to their ROI.

ROI is an input into the creation of the product roadmap and the determination of the cutlist, but it is not the only input. I always like to ensure that there is a healthy amount of time spent on engineering excellence (site stability, paying down technical debt, reducing latency, library upgrades, increased test coverage, improved tooling). I also like to ensure that we have a customer focus. Many of our features come directly from customer feedback via focus groups, quantitative surveys, and app store feedback; the Wishlist feature was one of these. I also reserve a healthy amount of time for strategic initiatives that may not provide lift in the short term but that set us up for success in terms of the Groupon 2020 vision.

Dilbert

Image source: Dilbert.com

Data-Driven Features

At Groupon we are lucky to have vast amounts of data that we can use to deliver a delightful product to our customers. We have worked with one million merchants to date; we have pumped more than $18 billion into local businesses; we have more than 1 billion Groupons sold; our app has been downloaded 171 million times, and we have saved customers more than $28 billion.

The Groupon platform handles tens of billions of user actions per month, and for machine learning algorithms that drive core product features our platform needs to make decisions (such as which deal to show the user next) in fractions of a second.

Developing product features that take advantage of these vast amounts of data in a performant way is an interesting challenge.

We use machine learning algorithms in a variety of ways to develop products here at Groupon:

  • Supply intelligence – There are millions of merchants we could call at any time to get onto our platform; how do we pick the best ones?
  • Fraud prevention – Fighting the bad guys in realtime.
  • Discovery and personalization – Selecting which deals to show a given user in her mobile app deal feed.
  • Image recognition – Identifying the best user-generated images with neural networks.
  • Logistics – Getting ahead of the order rush by sending extra inventory to the right warehouse in advance of high demand.
  • Customer support – AI-based chatbots to respond to and resolve customer issues instantaneously.

Groupon Mobile App

To make developing data-driven products faster, we built a generic, extensible machine learning platform at Groupon called Flux. Flux is the “Rosetta Stone” between data scientists and engineers.

Flux capaciter

Image: Wikimedia Commons

Data scientists work primarily in R. Flux models are written in Java and Clojure for stability and speed. Python is the glue that connects R and Java. It all runs on Groupon’s large Hadoop cluster.

To make the process for productionalizing machine learnings more robust, Groupon has an ETL management platform called Quantum Engineered Data (QED). QED reads from any source, and includes built-in data cleaning, error correction, and anomaly detection. Clean data is preserved and made available as a “feature catalog.” QED handles failures smartly, supporting falling back to yesterday’s model when appropriate. QED is able to plug into any source of truth—including streams, warehouse tables, and JSON endpoints.

Smart people doing interesting work

QED gives us a lot more confidence in the robustness of our models. In general, subtle changes to a single data field can seriously impact model performance, and nuances in the data set could look fine to tests but fail in the real world.

machine learning - xkcd

Image credit: XKCD

Monitoring

This blog post would be incomplete without a brief discussion of Groupon’s monitoring tools. We have a healthy suite of realtime alerts on product and engineering KPIs. We use splunk for logging and wavefront for graphing. Each service is staffed with a 24/7 on-call schedule, with escalation handled by pagerduty.

Additionally, each product area and business has an Amazon-style Weekly Business Review, where we look at metric trends longitudinally, identify areas of change or concern, and begin deep dives where appropriate.

The data warehouse uses Teradata and Apache Hive.

Experimentation

There are 100 teams at Groupon that run experiments. At any given time, around 200 experiments are being run simultaneously on the Groupon platform.

Groupon has a dedicated team called Optimize that built a bespoke tech platform for running product experiments with mathematical rigor. The experimentation platform is called Finch Express. Finch Express is built with Ruby on Rails, Node.js, Ember.js, Python, R, and Hadoop/Hive. The team has filed three patents for its innovations on product experimentation.

Essentially, Finch Express uses a technique called Group Sequential Analysis, first developed by Abraham Wald in 1945. Group Sequential Analysis has been used extensively in high-risk clinical trials, such as heart valve studies, where it’s possible that one treatment is actually harming the patients. Ethically, we would want to stop a harmful clinical trial immediately—but statistically, checking the experiment results mid-run or “peeking” will vastly increase your rate of false positives and invalidate your statistical results.

Group Sequential Analysis provides a controlled, statistically rigorous way to “peek” at experiment results at set points during the experiment run. This allows Groupon to end an experiment early if it is losing money, and to roll out an experiment early if it is deemed an early winner (capturing more upside).

Finch Express does all of this automatically. Product managers create the experiment in Finch Express, add a description and screenshots (to save the details for future product managers to reference), and launch the experiment at 50/50. Finch Express does the heavy lifting of dynamically determining the appropriate lift sensitivity for the experiment (based on traffic and conversion rate), performing the Group Sequential Analysis calculations, deeming the experiment a “success,” “failure,” or “flat” (most experiments end flat), and even automatically rolling out or rolling back the experiment based on its results. Then, Finch Express reports on the financial results of the experiment. The experimentation platform prevents product managers from statistical no-nos, such as peeking, unbalanced bucketing, and concluding the experiment too early. As a result, our experimentation processes have a high degree of statistical rigor.

On average, Group Sequential Analysis allows us to conclude experiments an average of 57.53% earlier compared to simply running them to a single final checkpoint. This reduces the cost of failed experiments, hastens upside capture of successful experiments, and allows for much faster iteration and innovation.

To date, Groupon product managers have run a total of 2,500 experiments, thanks in large part to the proprietary and patent-pending experimentation platform.

correlation - xkcd

Image credit: XKCD

Conclusion

Thanks for staying with me until the end! I hope this gives you an idea for how we use big data at global scale here at Groupon to create our product roadmaps, to innovate with new products and features, to monitor product performance, and to evaluate the impact of new initiatives. If you’re interested in learning more about data-driven product management opportunities at Groupon, have a look at our open roles.

Laura heads up product for consumer web, international, and LivingSocial at Groupon. She has a bachelor’s in mathematics from the University of Chicago and a master’s in computer science with specialization in machine learning from Georgia Tech. She has more than 10 years of experience in ecommerce product management at four Chicago tech companies, from early stage startups to publicly traded global companies. She is passionate about using analytics and machine learning to create a delightful customer experience.

Laura Hamilton

Group Product Manager

A Leader’s Garden—Designing a thriving organization when you aren’t in control.

A Leader’s Garden—Designing a thriving organization when you aren’t in control.

TECH PAPERS

Aug 19, 2014

A Leader’s Garden—Designing a thriving organization

when you aren’t in control

Written by Todd Webb, Senior Technical Leader

What if organizations were natural ecosystems, and like any other part of nature, cannot be controlled?

We like to think that we stand apart from nature, but what if we do not? What if our organization is just like a garden, with all its living and non-living elements, each at the same time independent and interdependent? How would this inform the way leaders approach the challenge of building and maintaining an organization?

If organizations were ecosystems then leaders must act as gardeners. Just as the master gardener must tend to an infinitely complex natural ecosystem, the leader must tend to an infinitely complex human ecosystem. Though the complexity of the ecosystem may be beyond our comprehension, if we understand its fundamental properties, we can help it thrive without the illusion of control.

What are the fundamental properties of a thriving organization?

A thriving organization is fundamentally a network of people with diverse motivations, sharing a purpose and objectives, and acting heedfully via systems and patterns of feedback.

A network of people

When you envision your organization do you think of a hierarchy? If you do, you are missing nearly everything important about how your organization works. If you were to visualize the real interactions between people inside and even outside the organization, you would see a tangled web of connections, some formal, some informal, some known, some unknown. You find people who influence the organization far more than their title or authority would suggest and others who have far less influence than you would expect given their positional power.

Although we may influence, we cannot control the network’s form, or evolution, but through an awareness of the network, can unleash its power.

(For more on networks watch The Power of Networks)

With diverse motivations

When you envision the motivations of the people in your organization do you think of everyone pulling the same direction with similar motivations or acting in self-interest and pulling against one another? Humans are wonderfully diverse and messy. They often have motives in line with organization goals, but they also have power motives, career motives, fear motives, outside interest motives.

Although we cannot control people’s motives, we can influence those motives through understanding and an expression of empathy. We must amplify shared motives and help people accept and find strength and creativity in diverse motives.

(To learn more about what happens when we fail to understand diverse motivations and act with empathy read Leadership and Self-Deception)

Sharing a purpose and objectives

When you ask anyone in your organization what the shared mission is, can they recite it? Do they list the organization’s values and principles? Do they share stories about people who embody those values and principles?

A shared understanding of values provides the glue for the foundation of the organization. This shared understanding of mission, values, principles, and objectives must permeate the organization and become a common lexicon within the company.

Acting heedfully

When you look around at the people working in your organization, are they aware of what is going on around them? If something went horribly wrong how quick would they notice and how well coordinated would their response be?

We must design and foster patterns of heedfulness. Heedfulness leads to resilient teams that can overcome significant chaos and change to reach a goal.

(To learn more about resiliency and heedfulness read Managing the Unexpected: Resilient Performance in an Age of Uncertainty)

Via systems and patterns of feedback

Keeping all those fundamental properties in mind, we must tend to the organization by designing and encouraging systems and patterns of feedback that foster heedfulness, and harness diversity, while reinforcing shared purpose and objectives.

Designing in Systems and Patterns

A master gardener understands the underlying patterns of the garden. They understand how each element of the garden produces, uses, or affects the nutrients, sunshine, water, temperature and other factors that help the garden thrive. They design their garden to reinforce those patterns with no illusion that they are in control. Master gardeners are systems thinkers.

We must be systems thinkers too. We must understand and design in systems and patterns of feedback. A thriving organization has many layers of feedback:

  • An all-hands Q&A with senior executives
  • A personal pat on the back for a teammate
  • A practice community that shares knowledge about the Java programming language
  • A book club that studies management practices
  • A governance meeting that helps an engineering team decide what they should work on
  • Software developers pair programming
  • Employees job shadowing to learn what it’s like for someone in a different department
  • A retrospective meeting
  • An always-on group chat room for a project team
  • A team standup meeting at the beginning of each day
  • An employee survey
  • A weekly 1-on-1 coaching session

The next time you think about how to influence your organization, to build great culture, to do great things—think about systems and patterns of feedback. Know that you are not in control, but if you design and foster the right systems and patterns of feedback you can help your organization thrive.

(To learn more about systems thinking read The Fifth Discipline)

Groupon is a massive data-driven experiment — this team helps run it

Groupon is a massive data-driven experiment — this team helps run it

Groupon is a massive data-driven experiment — this team helps run it

via BuiltIn Chicago

Groupon has tweaked and tested every corner of its e-commerce platform to find out precisely what makes customers click. Its platform is one of the world’s most optimized online destinations, but Groupon is still running daily experiments to add new features that increase business — and get rid of features that don’t.

Comprised of both data scientists and engineers, the Optimize team has built a tech platform for running those experiments with scientific rigor. We spoke with three team members about their efforts to reshape how Groupon thinks about data.

OPTIMIZE AT A GLANCE

TEAM DISCIPLINES: Engineers and data scientists.

WHAT THEY DO: Build tools that help Groupon iterate and improve on its customer experience; promote data literacy across the organization.

WHAT THEY DID BEFORE: The team includes statisticians, an astrophysicist, a music major, an economist and a theological studies major.

DID I READ THAT RIGHT?: Yep.

DEEPLY ROOTED: Optimize was one of Groupon’s founding teams.

INSPIRED BY: Pharma research, which pioneered a scientific way to “peek” at data and end experiments early.

THE STACK: Ruby on Rails, Node.js, Ember.js, Python, R, Hadoop/Hive.

What does Groupon’s Optimize team do?

Kristi Angel, data scientist: Experimentation is essential to product development at Groupon, and Optimize owns the platform for experimentation. Our software is used by product managers in developing new features and improvements on web and mobile platforms. We automate A/B testing, ensuring statistical rigor and provide reporting on experiment results.

I research statistical methodology to implement as features in our software, partner with product managers and leadership to develop a culture of statistical literacy, assist with the interpretation of experiment results and work with product analysts to design sound strategies for experimentation that does not fit within our framework. I also work on special projects related to the analysis of anomalous results and validation of data quality in our pipeline, for example.

Robb Broome, senior software engineer: The driving force is finding out whether the changes we make to our website are benefiting the business or hurting the business. As time goes by, we get better at finding solutions and ways of measuring impact that are more reliable and consistent.

What technologies power your experimentation platform?

David Oliver, engineering and product manager: Groupon’s website is built on a series of Node.js microservices, and our code is embedded as modules in those apps.

So, one of those Node modules that we own is Finch.js, which does two essential things: it buckets users into control or treatment groups, and it sends a message that the user saw the experiment. That message flows through a REST API, into Kafka and finally into Hadoop. We also have two Ruby on Rails applications. One is focused on consuming that data from Hadoop and piecing out the data in such a way that we can easily read it, and the other doubles as experiment configuration and the stats engine that powers our platform. Finally, we have an Ember.js app that our stakeholders use to create and manage experiments, and display experiment results.

If an experiment goes poorly, that can have a real impact on your bottom line. How do you balance those concerns against the need for statistical rigor?

Oliver: We use a technique called group sequential analysis, which was pioneered doing heart valve studies in the 1970s.

Angel: Imagine a clinical trial in which an experimental treatment is actually harming people. Ethically, one would want to stop the trial immediately, but statistically it is a bad thing to watch your data as it accumulates. All measurements have a natural variation and this randomness can easily be mistaken for a signal. We really only want to analyze the data once we know we have enough observations to be confident that what we are seeing is signal and not noise.

Group sequential analysis is a way to “peek” at the data in a controlled way, periodically checking in on the experiment in a statistically rigorous manner that limits false positive results. It turns out, a life-saving mechanism in pharma research is a revenue-saving mechanism at Groupon. We can more quickly capitalize on features that increase revenue and limit exposure to features that lose revenue.

What are the most interesting technological problems your team is solving?

Angel: We are working on arriving at the optimal attribution models across different areas of our business. For example, the attribution of a purchase to a specific experience — a home page feature, an email, a push notification — likely has a different window of time where we can reasonably say a specific purchase is a result of a specific experience.

Features that reduce friction on a site are likely to have a more immediate effect, whereas an email campaign might have an impact over a longer time frame. Mathematically, how do we find that optimal window? Technically, how do we implement and support a number of attribution methods for our framework?

Smart people doing interesting work

Groupon has a long history of experimentation, and a lot of data to work with. Does that present unique challenges?

Broome: Groupon is already highly optimized, which means you need better science and better math. In the olden days, it was easy to make huge changes that you didn’t need very sophisticated systems to see. We also have to pay very close attention to whether the data is getting in on time, because there are billions of transactions coming in, and the paths are pretty complicated.

What is one important change to how Groupon works that has emerged from the Optimize team’s work?

Angel: Finch Express, which is what we call our platform, has changed the way Groupon does A/B experimentation. Before its existence, experimentation was a pretty loose concept. Experimenters would monitor results day over day and run things until they looked “good” — usually meaning that the data was susceptible to the experimenter’s biases.

Today, we do not display data about the metric of interest until the experiment’s conclusion. That prevents experimenters from checking the results before enough data has accumulated and limits false conclusions.

How does that impact Groupon’s approach to data?

Angel: Our platform further moves the needle by shifting our organizational culture. It isn’t hard nor is it enough to be data driven. We must also be data literate. By building out a sophisticated platform and committing to comprehensive support for our consumers, we grow data literacy in the organization and as a result we’re able to make better decisions.

What drew you to Groupon?

Oliver: The Optimize team and getting to work both with programming languages I was already familiar with, like Ruby and JavaScript, as well as getting to touch other languages like Scala and Clojure. Our team doesn’t use Scala or Clojure anymore, but I love that Groupon as a whole isn’t afraid to try different languages and technologies — and just as easily move on if they’re not working. I was also attracted to having lots of moving parts and getting to work with all the different teams in the company.

What’s your favorite thing about Groupon’s culture?

Angel: I always felt supported in my personal development. Leadership here has a strong interest in making sure that you’re doing the kind of work you want. And although people are really smart, they’re also really fun. There’s a pervasive sense of humor across the company that I really appreciate, because I’m kind of a goofy person.

What does your team look for in developers?

Angel: We obviously look for technical skills; coding ability matters. Our team works in Ruby and Javascript and is full stack. But a great candidate might actually be a Java developer or have limited experience with front end development. We’re also looking for someone who is curious and humble, with good analytical intuition, and who has a strong sense of ownership, good communication skills and a commitment to excellence.

Oliver: It’s important to us that people are good communicators and who can explain complex concepts in a clear and straightforward way. We also look for professionalism, which is one of those qualities that’s hard to describe, but you know it when you see it. You have to be able to represent the team to the rest of the company.

Modularization of Android Apps

Modularization of Android Apps

Modularization in Android Apps

The mobile team organized a Meetup yesterday in Palo Alto out of the new large Spontaneous Combustion conference room. We had about 30 engineers from the area attend plus a great turnout from our team. Eric Fararro introduced Groupon engineering as a whole, followed by a technical talk about application modularization / instant app preparation given by Aolei Zhang and Erik Kamp. We fielded questions about this topic after the talk and had a handful of engineers interested in joining the Groupon mobile team as a result!

Special thanks to Stephane Nicolas for mentoring us through the talks, Daniel Molinero and Cody Henthorne for all the great feedback and pointers, and Lupe Leon for all the help organizational-wise.

Architecture Patterns for Backends beyond SOA

Architecture Patterns for Backends beyond SOA

Architecture Patterns for Backends beyond SOA

Javier Cano, Senior Software Engineer
Sergey Burkov, Senior Java Developer
December 13, 2017

In the Merchant Experience team specifically, and in Groupon in general, we have to deal with the challenge of scale and performance that our global business imposes. We make heavy use of SOA and microservices in our platform, though that is usually not enough. The solutions that we need make us explore and try different architectural patterns that move beyond what a SOA approach can provide. In this short talk we’ll explore some of these alternatives architectures, which problems they solve and how they integrate in microservices platform.

You can see lots more video of Grouponers and their smart friends on our YouTube channel.

Messaging at (Groupon) scale

Messaging at (Groupon) scale

Messaging at (Groupon) Scale

Nikita Berdikov
Senior Software Engineer
December 13, 2017

Every company is using messaging one way or another. So do we at Groupon. Messaging platform allows distributed heterogeneous services communicate with each other in asynchronous publish-subscribe fashion. Let’s talk about problems it helps to solve and problems it creates (especially from the owners of messaging infrastructure point of view). In addition we will go through tools we have built around messaging for better monitoring, maintenance and issues.

You can see lots more videos from Grouponers and other smart people on out YouTube channel.

Geekfest: Web.js (Full Stack Javascript)

Geekfest: Web.js (Full Stack Javascript)

Web.js (Full Stack Javascript)

Jaime Garcia Diaz
Software Engineer
November 14, 2017

1
2
3
4
5
6
7
Javascript is one of the most popular programming languages.
It's flexibility has impacted the way the web is being built.
Let'
s build a full-stack application with Javascript.
We'll touch on integrating with Docker, Mongo, Nextjs, Graphql, React and MaterialUI.
Recommended for anyone interested on Javascript and how it can be used on different web architecture tiers.
Demo:
https://github.com/jgarciadiaz/demo-events

See all Geekfest videos from Groupon and our friends.

Data Driven Chicago (Second Edition)

Data Driven Chicago (Second Edition)

Data Driven Chicago

Ilhan Kolko (Echo)
Andrew Lisy and Laura Hamilton (Groupon)
Tyler Hanson (Reverb)
Mary Feigenbutz and Greg Reda (Sprout Social)
Laurie Skelly and Elizabeth Cleveland (Trunk Club)
Moderated by Alli Diedrick (Built In Chicago)
November 2, 2017

Showcasing some of the great data-driven and machine-learning talent here in Chicago. Brought to your by Groupon, hosted by Built In Chicago.

See more videos from Groupon and our friends.

Grox: The Art of the State

Grox: The Art of the State

Grox: The Art of the State

Alin Turcu
Android Software Engineer
November 2, 2017

Once upon a time, you started a new app and everything was simple and nice: a few features, a simple UI and that’s it. But then it became bigger and bigger, and the logic became more complex, more entangled. Suddenly, you have a database, you have multiple network calls, several components talking to each others on different threads, callbacks everywhere and multiple user interactions.

The state of your app is modified from everywhere and at any time. At this point you can’t even clearly say in which state your app is after a user interacts with a few elements in the UI.

Let’s consider a typical Android App

Our apps do things like interacting with a server, with the local disk storage, and the UI. All these interactions can change the state of our app and can happen on different threads. How can we easily coordinate all this without losing our clear state?

Grox

Grox is an open source library inspired by Flux, Redux, Cycle, and Rx Managed State that makes it easier to manage the internal state of complex applications.

Let’s see how the above diagram would change with Grox.

Wow… That’s a lot of new stuff!

Let’s walk through all the elements in the diagram.

Grox concepts

Grox comes with 3 different modules.

The “Grox-Core” module defines the Action and Store classes.

The Store is the component that contains the state of your app. It is the unique source of truth about its state. The Store keeps state, dispatches actions and notifies its listeners of state changes. This part of the library is synchronized so only one action can change the state of the store at a time.

There is a second variant of the Grox-Core module, called “Grox-Core-Rx“ which uses Observable instead of a list of listeners.

Actions change the state inside the store, and in turn the store will emit the new state. Actions are pure functions, 100% reproducible and testable, their only purpose is to create a new state from the old one. The above diagram exemplifies how the store and actions work together. Actions are generated by commands.

Commands produce a stream of Actions. They are part of the Grox-Commands module and are based on RxJava. Commands return a stream of actions represented by an Rx Observable. Every possible output of the command is mapped to an Action. Commands are used in Grox to interact with non-pure logic, not 100% reproducible parts of your app such as a network call, an interaction with the file system, etc. Events are generated by user interactions or by system evens like push notification or location change. They are mapped to commands that get triggered when an event occurs.

How to Grox

Let’s use the concepts described above to create a login screen.

The sequence of events would be something like this:

Let’s see what happens when the user presses the login button:

  1. A click event is generated.
  2. A command will be started that will make a HTTP call to our server. This command knows the sequence of steps to be executed for the “Login use case”
  3. The first action the command will emit will create a new state that will represent the “login in progress state”.
  4. Once the HTTP call is completed, the command will emit a new Action that will change the state from “login in progress state” to “login finished successful” state.

The app will receive the new state from the store by subscribing to a stream of states. These states will be used to change the UI of your app. This way the UI becomes a passive representation of the state.

Enough talking, show me code!

Grox works wonderfully with RxJava so we will also use it in our example. Grox does not impose the use of RxJava, we can use the Store and Actions without it but we would need RxJava if we want to use commands.

First we define our login states.

Next, we build the initial state of the store. In the initial state we have no logged in user and the login request is not started.

Let’s see how the Login command looks like:

In the above example we assume that we have a LoginApiClient class that performs the actual network request and throws an error if the request fails or returns an error code different than 200. We perform the request in a separate class and we pass it as a parameter so we are able to test this command by mocking the loginApiClient object. The LoginApiClient returns the logged in user if the request is successful

All the outputs of a command must be actions, even in the case of errors. At the network layer, errors do occur but in upper layers, they will represent a state of the app. In case of an error, a new “error state” is created and the UI can display it accordingly.

Using the RxChain, we map the successful result of the request to a LoginSuccessfulAction and in case of an error to a LoginFailedAction. We also start with a LoginInProgressAction so, we change the state to “in progress” which will make the UI display a progress indicator while the actual network request will be performed.

The LoginSuccessfulAction and the LoginInProgressAction are very similar, their responsibility is to create a new state using the old one.

LoginSuccessfulAction saves the logged in user in the store so that the UI can display the user details.

The LoginInProgressAction works exactly in the same way. It just sets the loginState to LOGIN_IN_PROGRESS. Pretty straightforward, right? LoginFailedAction should also look familiar by now.

As you can see in the example, actions are pure functions and are easy to test. They are 100% reproducible (so, no network calls, no disk access, etc. as they can fail). The non-reproducible part of the job of our app goes into commands.

Wiring everything together

Ok. We have built all our components. Every component is encapsulated, testable, and has a clear responsibility. The next thing we need to do is to wire everything together. This can be done inside the View or inside the Presenter, it depends on what kind of design you have inside the app. For simplicity in our example we wire everything inside the LoginActivity.

Let’s walk through what is happening here. First we subscribe for store updates. states is a static method in Grox library that receives the store as parameter and returns a stream of states. We observe these states on the main thread and we update the UI once a new state is emitted.

Next, for every click event, we create a LoginCommand and we “launch” it by calling the actions() method. This method will return a stream of Actions that will be dispatched to the store. If we use RxBindings the code becomes even more elegant:

Now you can clearly see that the click event is mapped to a LoginCommand. That’s it! Pretty simple, right? We can now apply this design for every event and every operation in your app.

Once you start to Grox, you never go back

Smart people doing interesting work

Grox your Android app!

Grox provides a clean solution for managing the state of an app. The main benefits of using the Grox design are:
Clear Concepts: Every entity described above has a well defined role. Grox provides a clean architecture, with a clear separation of concerns between state, store, commands, events.

  • Unified Process: Using Grox design you can actually represent everything that can change the state of your app or of a certain screen. All parts of an app, all screens, can use Grox to unify their handling of state. You can represent user interaction and you can represent operations. These operations can have different purposes like accessing the file system, making network request or even performing some computations.
  • Scalable: You can add as many operations as you want and your app state will still be clear at any time. The same design easily scales to all events and for every operation that changes your app’s state.
  • Testable: All the components that you create using Grox can be tested.

More info:

Github: https://github.com/groupon/grox

Cool introduction video:

Thanks to Stephane NICOLAS, keith smyth, Cody Henthorne, Eric Farraro, Samuel Guirado Navarro, David Dumitru, and Mihaly Nagy.