Capital One
Reducing cloud costs
Helping developers effectively reduce cloud costs in their coding environments with minimal friction
Role: Designer and researcher
Date: Q2 2024
-30%
notebook costs one month post-launch
~$1M
annual notebook
cost savings
~$11M
annual cost savings from combined features
What are notebooks?
At Capital One, developers use tools like Jupyter Notebooks to write, test and experiment with code to train machine learning models and develop GenAI applications.
Each notebook size is configured in an internal developer platform called AIML Studio, used everyday by engineers and data scientists.
Problem
Developers were inefficiently using notebooks. Their associated business units pay for all cloud compute they request, whether or not they fully use it.
The product team analyzed data that showed costly usage patterns from launching coding environments in AIML Studio.
Average notebook utilization was low, only 8.5%.
Half of notebook costs came from notebooks with <5% utilization.
Research
Approach and objective: Early discovery and mid-project concept testing helped understand developers’ mental models and validate lo-fi design concepts.
Findings:
Most participants didn’t actively think about right-sizing notebooks because they didn’t see clear and convenient signals about how to do so and the consequences.
Participants strongly preferred seeing both cost and usage at a quick at-a-glance summary level and at the individual notebook level, to drive behavior and right-size notebooks.
Next steps:
Following research, I ran a workshop to help align with the PMs and engineering teams towards a single design direction from multiple solutions: a cost dashboard on the Notebooks homepage.
Button: download usage metrics
Jupyter Notebook UI widget with
real-time usage metrics
Metrics dashboard
Process
Early visualization explorations included bullet graphs and progress bars.
Progress bar
Bullet graph
Final designs had the following features:
1
Summary accordion gives at-a-glance information that applies to all user notebooks, even when collapsed
2
Additional information to indicate lack of real-time data
3
Dropdown accordion pattern chosen over card to progessively disclose more details as needed, reducing user overwhelm
4
Critical badge styled with high emphasis to draw attention to potential notebook failures and lead to immediate action
4
3
2
1
Impact
Notebook costs reduced by 30% across the platform one month after implementation.
The cost dashboard has created ~$1M in annual cost savings alone. Annually, business units at Capital One collectively save ~$11M from multiple cost-saving features.
Notebook spend across business units
May
2024
June
Notebooks dashboard cost savings (~$1M)
Total cost savings (~11M)
Total annual average compute spend (~$37M)
Capital One
Reducing cloud costs
Helping developers effectively reduce cloud costs in their coding environments with minimal friction
Role: Designer and researcher
Date: Q2 2024
-30%
notebook costs one month post-launch
~$1M
annual notebook cost savings
~$11M
annual cost savings from combined features
What are notebooks?
At Capital One, developers use tools like Jupyter Notebooks to write, test and experiment with code to train machine learning models and develop GenAI applications.
Each notebook size is configured in an internal developer platform called AIML Studio, used everyday by engineers and data scientists.
Problem
Developers were inefficiently using notebooks. Their associated business units pay for all cloud compute they request, whether or not they fully use it.
The product team analyzed data that showed costly usage patterns from launching coding environments in AIML Studio.
Average notebook utilization was low, only 8.5%.
Half of notebook costs came from notebooks with <5% utilization.
Research
Approach and objective:
Early discovery and mid-project concept testing helped understand developers' mental models and validate lo-fi design concepts.
Findings:
Most participants didn't actively think about right-sizing notebooks because they didn't see clear and convenient signals about how to do so and the consequences.
Participants strongly preferred seeing both cost and usage at a quick at-a-glance summary level and at the individual notebook level, to drive behavior and right-size notebooks.
Next steps:
Following research, I ran a workshop to help align with the PMs and engineering teams towards a single design direction from multiple solutions: a cost dashboard on the Notebooks homepage.
Button: download usage metrics
Jupyter Notebook UI widget with real-time usage metrics
Metrics dashboard
Process
Early visualization explorations included bullet graphs
and progress bars.
Progress bar
Bullet graph
Final designs had the following features:
1
Summary accordion gives at-a-glance information that applies to all user notebooks, even when collapsed
2
Additional information to indicate lack of real-time data
3
Dropdown accordion pattern chosen over card to progessively disclose more details as needed, reducing user overwhelm
4
Critical badge styled with high emphasis to draw attention to potential notebook failures and lead to immediate action
Impact
Notebook costs reduced by 30% across the platform one month after implementation.
The cost dashboard has created ~$1M in annual cost savings alone. Annually, business units at Capital One collectively save ~$11M from multiple cost-saving features.
Notebook spend across business units
May
2024
June
Notebooks dashboard cost savings (~$1M)
Total cost savings (~11M)
Total annual average compute spend (~$37M)
Capital One
Reducing cloud costs
Helping developers effectively reduce cloud costs in their coding environments with minimal friction
Role: Designer and researcher
Date: Q2 2024
-30%
notebook costs one month post-launch
~$1M
annual notebook cost savings
~$11M
annual cost savings from combined features
What are notebooks?
At Capital One, developers use tools like Jupyter Notebooks to write, test and experiment with code to train machine learning models and develop GenAI applications.
Each notebook size is configured in an internal developer platform called AIML Studio, used everyday by engineers and data scientists.
Problem
Developers were inefficiently using notebooks. Their associated business units pay for all cloud compute they request, whether or not they fully use it.
The product team analyzed data that showed costly usage patterns from launching coding environments in AIML Studio.
Average notebook utilization was low, only 8.5%.
Half of notebook costs came from notebooks with <5% utilization.
Research
Approach and objective:
Early discovery and mid-project concept testing helped understand developers’ mental models and validate lo-fi design concepts.
Findings:
Most participants didn’t actively think about right-sizing notebooks because they didn’t see clear and convenient signals about how to do so and the consequences.
Participants strongly preferred seeing both cost and usage at a quick at-a-glance summary level and at the individual notebook level, to drive behavior and right-size notebooks.
Next steps:
Following research, I ran a workshop to help align with the PMs and engineering teams towards a single design direction from multiple solutions: a cost dashboard on the Notebooks homepage.
Button: download usage metrics
Jupyter Notebook UI widget with real-time usage metrics
Metrics dashboard
Process
Early visualization explorations included bullet graphs and progress bars.
Progress bar
Bullet graph
Final designs had the following features:
1
Summary accordion gives at-a-glance information that applies to all user notebooks, even when collapsed
2
Additional information to indicate lack of real-time data
3
Dropdown accordion pattern chosen over card to progessively disclose more details as needed, reducing user overwhelm
4
Critical badge styled with high emphasis to draw attention to potential notebook failures and lead to immediate action
Impact
Notebook costs reduced by 30% across the platform one month after implementation.
The cost dashboard has created ~$1M in annual cost savings alone. Annually, business units at Capital One collectively save ~$11M from multiple cost-saving features.
Notebook spend across business units
May
2024
June
Notebooks dashboard cost savings (~$1M)
Total cost savings (~11M)
Total annual average compute spend (~$37M)