Ángel Alex ander CabreraI am a PhD candidate in the Human-Computer Interaction Institute (HCII) at Carnegie Mellon University, advised by Adam Perer and Jason Hong. I work on human-centered AI, specifically using techniques from HCI and visualization to help people better understand and improve their AI systems. I am supported by an NSF Graduate Research Fellowship and have spent time at Apple AI/ML, Microsoft Research, & Google. | |
---|---|
Education | |
2024 | Ph.D. in Human-Computer Interaction (HCI)Carnegie Mellon UniversityAdvised by Adam Perer and Jason Hong. |
2022 | M.S. in Human-Computer InteractionCarnegie Mellon University |
2019 | B.S. in Computer ScienceGeorgia Institute of TechnologyConcentration in intelligence and modeling/simulation. Minor in economics. |
Fall 2017 | Sciences Po - Paris, FranceExchange program with a focus on economics and political science. |
Work Experience | |
April 2024 - Present | StealthFounding Member of Technical StaffAI + Bio. |
Summer 2021 | Apple AI/MLResearch InternModular machine learning interfaces, see Symphony. |
Summer 2020 | Microsoft ResearchResearch InternBehavioral model analysis, see AIFinnity. |
Summer 2018 | Software Engineering InternAutomated driver assistance and hyperlocal weather prediction for Android Auto. |
Summer 2017 | Software Engineering InternAnomaly detection and regression analysis system for Google's data processing pipelines. |
Summer 2016 | Engineering Practicum InternAnalytics platform for monitoring and detecting erroneous edits to Google Maps. |
Awards | |
2023 | Mozilla Technology Fund$50,000 grant to develop Zeno as an auditing tool for AI. |
2023 | Stanford HAI Audit ChallengeZeno was a finalist for the HAI challenge for designing better AI auditing tools. |
2019 - 2022 | National Science Foundation Graduate Research Fellowship (NSF GRFP)Three-year graduate fellowship for independent research. Full tuition with an annual stipend of $34,000. |
2019 | Love Family Foundation ScholarshipCo-awarded the $10,000 scholarship for the undergraduate with the most outstanding scholastic record. |
2015 - 2019 | Stamps President's ScholarGeorgia Tech and the Stamps Family Charitable FoundationFull ride scholarship with $15,000 in extracurricular funding awarded to 10 incoming students. |
2018 | The Data Open DatathonCorrelation One and Citadel SecuritiesPlaced third and won $2,500 for creating a ML system to predict dangerous road areas. |
Refereed Publications | |
[11] | Where Does My Model Underperform? A Human Evaluation of Slice Discovery AlgorithmsNari Johnson, Ángel Alexander Cabrera, Gregory Plumb, Ameet TalwalkarAAAI Conference on Human Computation and Crowdsourcing (HCOMP). Delft, Netherlands, 2023. |
[10] | Towards a More Rigorous Science of Blindspot Discovery in Image Classification ModelsGregory Plumb*, Nari Johnson*, Ángel Alexander Cabrera, Ameet TalwalkarTransactions on Machine Learning Research (TMLR). 2023. |
[9] | Zeno: An Interactive Framework for Behavioral Evaluation of Machine LearningÁngel Alexander Cabrera, Erica Fu, Donald Bertucci, Kenneth Holstein, Ameet Talwalkar, Jason I. Hong, Adam PererACM Conference on Conference on Human Factors in Computing Systems (CHI). Hamburg, Germany, 2023. |
[8] | Improving Human-AI Collaboration with Descriptions of AI BehaviorÁngel Alexander Cabrera, Adam Perer, Jason I. HongACM Conference on Computer-Supported Cooperative Work and Social Computing (CSCW). Minneapolis, 2023. |
[7] | What Did My AI Learn? How Data Scientists Make Sense of Model BehaviorÁngel Alexander Cabrera, Marco Tulio Ribeiro, Bongshin Lee, Rob DeLine, Adam Perer, Steven M. DruckerACM Transactions on Computer-Human Interaction (TOCHI). 2023. |
[6] | Symphony: Composing Interactive Interfaces for Machine LearningÁngel Alexander Cabrera*, Alex Bäuerle*, Fred Hohman, Megan Maher, David Koski, Xavier Suau, Titus Barik, Dominik MoritzACM Conference on Conference on Human Factors in Computing Systems (CHI). New Orleans, 2022. |
[5] | An open repository of real-time COVID-19 indicatorsAlex Reinhart, Logan Brooks, Maria Jahja, Aaron Rumack, Jingjing Tang, [et al, including Ángel Alexander Cabrera]Proceedings of the National Academy of Sciences (PNAS). 2021. |
[4] | Discovering and Validating AI Errors With Crowdsourced Failure ReportsÁngel Alexander Cabrera, Abraham Druck, Jason I. Hong, Adam PererACM Conference on Computer-Supported Cooperative Work and Social Computing (CSCW). Virtual, 2021. |
[3] | Regularizing Black-box Models for Improved InterpretabilityGregory Plumb, Maruan Al-Shedivat, Ángel Alexander Cabrera, Adam Perer, Eric Xing, Ameet TalwalkarConference on Neural Information Processing Systems (NeurIPS). Vancouver, 2020. |
[2] | Designing Alternative Representations of Confusion Matrices to Support Non-Expert Public Understanding of Algorithm PerformanceHong Shen, Haojian Jin, Ángel Alexander Cabrera, Adam Perer, Haiyi Zhu, Jason I. HongACM Conference on Computer-Supported Cooperative Work and Social Computing (CSCW). Virtual, 2020. |
[1] | FairVis: Visual Analytics for Discovering Intersectional Bias in Machine LearningÁngel Alexander Cabrera, Will Epperson, Fred Hohman, Minsuk Kahng, Jamie Morgenstern, Duen Horng (Polo) ChauIEEE Conference on Visual Analytics Science and Technology (VAST). Vancouver, Canada, 2019. |
Workshops, Demos, Posters, and Preprints | |
[5] | An In-depth Look at Gemini's Language AbilitiesSyeda Nahida Akter, Zichun Yu, Aashiq Muhamed, Tianyue Ou, Alex Bäuerle, Ángel Alexander Cabrera, Krish Dholakia, Chenyan Xiong, Graham NeubigPreprint. 2023. |
[4] | Evaluating Systemic Error Detection Methods using Synthetic ImagesGregory Plumb, Nari Johnson, Ángel Alexander Cabrera, Marco Tulio Ribeiro, Ameet TalwalkarICML - Workshop on Spurious Correlations, Invariance and Stability. Baltimore, MD, 2022. |
[3] | "Public(s)-in-the-Loop": Facilitating Deliberation of Algorithmic Decisions in Contentious Public Policy DomainsHong Shen, Ángel Alexander Cabrera, Adam Perer, Jason I. HongCHI - Fair & Responsible AI Workshop. Hawaii, USA, 2020. |
[2] | Discovery of Intersectional Bias in Machine Learning Using Automatic Subgroup GenerationÁngel Alexander Cabrera, Minsuk Kahng, Fred Hohman, Jamie Morgenstern, Duen Horng (Polo) ChauICLR - Debugging Machine Learning Models Workshop (Debug ML). New Orleans, Louisiana, USA, 2019. |
[1] | Interactive Classification for Deep Learning InterpretationÁngel Alexander Cabrera, Fred Hohman, Jason Lin, Duen Horng (Polo) ChauCVPR - Demo. Salt Lake City, Utah, USA, 2018. |
Talks | |
2023 | Authoring Interactive AI Evaluation Reports with ZenoLLMs in Production - Part III |
2022, 2023 | Evaluating Machine LearningCMU 05-618/318: Human-AI Interaction |
2022 | Visualization and Machine LearningCMU 17-428/728: ML and Sensing |
2022 | Designing Large Web ApplicationsCMU 05-431/631: Software Structures for User Interfaces (SSUI) |
2022 | Modern Web FrameworksCMU 05-431/631: Software Structures for User Interfaces (SSUI) |
2021 | Ethics in Data VisualizationCMU 05-899: Data Visualization |
2021 | D3 Deep DiveCMU 05-899: Data Visualization |
2021 | Data Science Widgets with Svelte and JupyterSvelte Summit 2021 |
Teaching | |
Fall 2022 | 05-431/631: Software Structures for User Interfaces (SSUI)Graduate Teaching Assistant @ Carnegie MellonTeach weekly lab sections, grade tests and homeworks. |
Fall 2021 | 05-499:C: Data VisualizationGraduate Teaching Assistant @ Carnegie MellonTaught a D3 course and led an ethics workshop in addition to grading and course management. |
2016 - 2018 | CS1332: Data Structures and AlgorithmsUndergraduate Teaching Assistant @ Georgia TechTaught a weekly recitation, graded tests and homework, and helped create assignments. |
Mentoring | |
Spring 2023 - Fall 2023 | Josh ZhouB.S. in Computer Science, Carnegie MellonInstance tagging for Zeno. |
Summer 2022 - Fall 2023 | Erica FuB.S. in Information Systems, Carnegie MellonUX design for an ML evaluation platform. See Zeno. |
Spring 2023 - Summer 2023 | Steven HuangResearch Associate, Carnegie MellonChart builder for Zeno. |
Fall 2022 - Spring 2023 | Tianqi WuM.S. in Computer Science, Carnegie MellonInteractive slice discovery for Zeno. |
Summer 2022 - Spring 2023 | Donny BertucciB.S. in Computer Science, Oregon State University. REU at Carnegie MellonInteractive model debugging. See Zeno. |
Summer 2022 | Kan SunB.S. in Math, Carnegie Mellon.Algorithmic discovery of ML errors. |
Fall 2021 - Spring 2022 | Emily GuoB.S. in Statistics and Machine Learning, Carnegie MellonImproving human-AI interaction with descriptions of model behavior. |
Spring 2020 - Spring 2021 | Abraham DruckB.S. in Mathematical Sciences, Carnegie Mellon. Now: Technology Analyst at Morgan StanelyCrowdsourced discovery of ML failures for image captioning. See Deblinder. |
2020 | CMU AI Mentoring Program |
Service | |
Organizer | |
2023 | CSCW Workshop on Supporting User Engagement in Testing, Auditing, and Contesting AI |
Program Committee | |
2022 - 2023 | AC, ACM Conference on Human Factors in Computing Systems (CHI) Late Breaking Work |
2022 - 2023 | PC, ACM Fairness, Accountability, and Transparency (FAccT) |
2022 - 2023 | PC, IEEE VIS Workshop on Visualization for AI Explainability (VISxAI) |
2023 | PC, CHI Workshop on Trust and Reliance in AI-Assisted Tasks (TRAIT) |
2022 | AC, ACM Conference on Computer-Supported Cooperative Work and Social Computing (CSCW) Posters |
Reviewer | |
2021 - 2024 | ACM Conference on Human Factors in Computing Systems (CHI) |
2021 - 2023 | ACM Conference on Computer-Supported Cooperative Work and Social Computing (CSCW) |
2020 - 2023 | IEEE VIS |
2022 - 2023 | ACM Fairness, Accountability, and Transparency (FAccT) |
2022 | ACM Symposium on User Interface Software and Technology (UIST) |
2022 | IEEE Computer Graphics and Applications (CGASI) |
2019 - 2021 | IEEE Transactions on Visualization and Computer Graphics (TVCG) |
2019 | ACM Transactions on Interactive Intelligent Systems (TiiS) |
Student Volunteer | |
2019 | IEEE VIS |
2019 | ACM Fairness, Accountability, and Transparency (FAccT) |
Department | |
2022 - 2023 | REU application reviewer |
2020 - 2021 | Ph.D. student faculty representative |
Press | |
2024 | "Google Gemini: Everything you need to know about the new generative AI platform" - TechCrunch |
2023 | "Google Gemini is not even as good as GPT-3.5 Turbo, researchers find" - Venture Beat |
2023 | "Auditing AI: Announcing the 2023 Mozilla Technology Fund Cohort" - Mozilla |
2020 | "New forecasting data could help public health officials prepare for what's next in the coronavirus pandemic" - CNN |
2020 | "Facebook and Google Survey Data May Help Map Covid-19's Spread" - Wired |
2020 | "Carnegie Mellon Unveils Five Interactive COVID-19 Maps" - Carnegie Mellon |
2020 | "Visualizing Fairness in Machine Learning" - Data Stories Podcast |
2019 | "Alex Cabrera Wins Love Family Foundation Scholarship" - GT SCS |
2019 | "Georgia Tech Satellite Successfully Launched Into Space " - Georgia Tech |
2018 | "Datathon Challenges Students to Create Solutions to Real-World Problems" - GT SCS |
Projects and Open Source | |
2023 | ZenoAn interactive ML evaluation framework for any data or model. |
2021 | Svelte + VegaA Svelte component for reactively rendering Vega and Vega-Lite visualizations. |
2021 | Svelte + Jupyter WidgetsA framework for creating reactive data science widgets using Svelte JS. |
2020 | COVIDCast Visualization of COVID-19 IndicatorsInteractive visualization system of COVID-19 indicators gathered through >20,000,000 surveys on Facebook and Google by CMU Delphi. |
2015 - 2017 | PROX-1 SatelliteFlight Software Lead and ResearcherLed a team of engineers in developing the software for a fully undergraduate-led satellite mission. |
2014 | CTF ResourcesGuide and resources for capture the flag (CTF) competitions with over 1.6k stars on GitHub. |
Selected Courses | |
Ph.D. | MultiModal Machine LearningCausality and Machine LearningHuman Judgement and Decision MakingApplied Research Methods |
B.S. | Deep LearningData and Visual AnalyticsMachine LearningComputer SimulationHonors Algorithms |
cabreraalex.com
cabrera@cmu.edu
Google Scholar
GitHub