| |
|
|
The Department of Computer Science
of Colorado State University, in cooperation with ISTeC
(Information Science and Technology Center), offers the CS
Colloquium series as a service to all who are interested
in computer science. When in-person meetings are possible, most seminars are scheduled for
Monday 11:00AM -- 11:50AM in CSB 130 or Morgan Library
Event Hall. For help finding the locations of our seminar
meetings, consult the on-line CSU campus map. |
Upcoming Events
CS Colloquium Schedule, Fall 2023
|
|
Computer Science Department Colloquium Introduction to the Graduate Program Speaker: Sanjay Rajopadhye, Professor and Graduate Director, Computer Science Department, Colorado State University When: 11:00AM ~ 11:50AM, Monday August 21, 2023  Abstract: Dr. Rajophadye introduces the Computer Science graduate program at CSU. |
|
|
|
Computer Science Department Colloquium No Seminar Speaker: Computer Science Faculty, Colorado State University When: , Monday August 28, 2023 Abstract: |
|
|
|
Computer Science Department Colloquium No Seminar Due to Labor Day Speaker: Computer Science Faculty, Colorado State University When: , Monday September 4, 2023 Abstract: |
|
|
|
Computer Science Department Colloquium CS Faculty Lightning Presentations of Current Research: Q&A, First session Speaker: Computer Science Faculty, Colorado State University When: 11:00AM ~ 11:50AM, Monday September 11, 2023 Abstract: CS Faculty answers questions about their research, and opportunities in their research group for students at CSU. |
|
|
|
Computer Science Department Colloquium Evaluation of Classification Models in Limited Data Scenarios with Application to Additive Manufacturing Speaker: Farhad Pourkamali, Assistant Professor of Mathematical and Statistical Sciences, University of Colorado Denver When: 11:00AM ~ 11:50AM, Monday September 18, 2023 Abstract: Scientific observations and experiments provide valuable data to build machine learning (ML) models that reveal links between input variables and quantities of interest. Specifically, adopting machine ML-based surrogate models in scientific and engineering applications can accelerate design space exploration and optimization where closed-form analytical models for complex systems are unattainable. In this talk, we present recent work on developing a novel framework that enables the generation of accurate and unbiased test loss estimates using fewer labeled samples, effectively evaluating the predictive performance of classification models in data-limited applications. Central to the framework's innovation is designing an adaptive sampling distribution, which identifies pivotal testing samples based on the dynamic interplay between learner and evaluator models. A noteworthy aspect of this adaptive distribution lies in its ability to continually recalibrate the supervisory role of the evaluator model by prioritizing inputs that exhibit disparities. Comprehensive experimental analyses on two sparse data sets from material extrusion additive manufacturing problems concerning filament and printer selection validate the framework's superiority over uniform and fixed sampling distributions. Bio: Farhad Pourkamali-Anaraki is an Assistant Professor in the Department of Mathematical and Statistical Sciences at the University of Colorado Denver (CU Denver). Previously, he was an Assistant Professor of Computer Science at the University of Massachusetts Lowell (2018-2022) and received his Ph.D. in Electrical Engineering from CU Boulder in 2017. His main research interest revolves around transitioning machine learning models from controlled lab environments to real-world settings involving unpredictable and changing conditions, such as quantifying aleatoric and predictive uncertainties. His research is currently funded by NASA and Army Research Lab (ARL) to accelerate the design and discovery of new materials in extreme conditions using cost-effective and reliable machine learning models. |
|
|
|
Computer Science Department Colloquium CS Faculty Lightning Presentations of Current Research: Q&A, Second session Speaker: Computer Science Faculty, Colorado State University When: 11:00AM ~ 11:50AM, Monday September 25, 2023 Abstract: CS Faculty answers questions about their research, and opportunities in their research group for students at CSU. |
|
|
|
The Information Science and Technology Center Distinguished Lectures When Robots Learn to Write, What Happens to Learning? Four Proposals for AI Tools in Teaching & Learning Speaker: Dr. Bill Hart-Davidson, Professor in the Department of Writing, Rhetoric, and American Cultures, Michigan State University When: 11:00AM ~ 12:00PM, Monday October 2, 2023 Abstract: The availability of AI and Large-Language Models in particular has rapidly become a disruptive force in education over the last few months. What happened recently to make these models more powerful and more widely accessible? What are the capabilities of these models and how can they change teaching and learning? In this session, I'll offer some responses to these questions from my point of view as a researcher and maker of writing technologies, and as a teacher and administrator. I will also offer four proposed changes for educators to consider at the pedagogy, curriculum, policy and ethics levels as we imagine our writing lives, together, with non-human agents. Each of these areas will be the basis for discussion and some hands-on creative work applicable to our contemporary classrooms and students in the workshop to follow. Bio: Bill Hart-Davidson, Ph.D., is a Professor in the Department of Writing, Rhetoric, and American Cultures, a Senior Researcher in the Writing, Information and Digital Experience (WIDE) Research Center and Associate Dean of Research and Graduate Education in the College of Arts & Letters at Michigan State University. He has published over 100 articles and book chapters and is co-inventor of Eli Review, a software service that supports peer learning in writing, feedback, and revision. Bill's research and teaching focus on writing and feedback in both school and professional settings. |
|
|
|
The Information Science and Technology Center Distinguished Lectures Genre Signaling: Lessons Learned from Teaching Robots (and People) to Communicate Science Speaker: Dr. Bill Hart-Davidson, Professor in the Department of Writing, Rhetoric, and American Cultures, Michigan State University When: 4:00PM ~ 6:00PM, Monday October 2, 2023 Abstract: This seminar will reflect on an emergent concept - genre signaling - that grew from several experiments to train machine learning classifiers to recognize scientific information. Genre signaling describes the communicative behaviors that help both humans and machines evaluate written discourse as being more or less accurate, reliable, and trustworthy to the degree it can be recognized as being, or perhaps more accurately, as doing science. In this talk, I'll focus on a few insights that my colleagues & I learned that may be useful for scientists who want to communicate with a broader audience about their own work or about scientific knowledge that is in the public interest. One example is the importance of "hedging" - a move to match the strength of a claim to the strength of available evidence that is essential in science but which can complicate messages intended for a broader public audience. Bio: Bill Hart-Davidson, Ph.D., is a Professor in the Department of Writing, Rhetoric, and American Cultures, a Senior Researcher in the Writing, Information and Digital Experience (WIDE) Research Center and Associate Dean of Research and Graduate Education in the College of Arts & Letters at Michigan State University. He has published over 100 articles and book chapters and is co-inventor of Eli Review, a software service that supports peer learning in writing, feedback, and revision. Bill's research and teaching focus on writing and feedback in both school and professional settings. |
|
|
|
CS Colloquium Series Learning Generalizable Neuro-Symbolic Abstraction Hierarchies for Robot Planning Speaker: Naman Shah, Arizona State University When: 2:00PM ~ 3:00PM, Wednesday October 4, 2023 Abstract: This seminar will reflect on an emergent concept - genre signaling - that grew from several experiments to train machine learning classifiers to recognize scientific information. Genre signaling describes the communicative behaviors that help both humans and machines evaluate written discourse as being more or less accurate, reliable, and trustworthy to the degree it can be recognized as being, or perhaps more accurately, as doing science. Bio: Naman Shah, is a PhD student candidate working in Autonomous Agent and Intelligent Robots (AAIR) lab directed by Dr. Siddharth Srivastava at Arizona State University, Tempe, USA. His research interest includes learning and using abstractions for sequential decision-making problems for robotics. He aims to learn hierarchical abstractions for robot planning tasks and use them to solve different problems such as hierarchical planning, reinforcement learning, and mobile manipulation in stochastic settings. |
|
|
|
Computer Science Department Colloquium Applied Perception in Extended Reality Speaker: Dr. Mohammed Safayet Arefin, Computer Science, Colorado State University When: 11:00AM ~ 11:50AM, Monday October 9, 2023 Abstract: The extended reality space can be divided into three environmental realities: real world, augmented reality (AR), and virtual reality (VR). In all three realities, information can be presented at different distances from the user. Sometimes users may need to integrate information from different depths by continuously switching visual attention and eye focus. For example, consider a scenario where a surgeon is using an optical see-through (OST) AR head-mounted display (HMD) while performing surgery. To perform the surgery, the surgeon needs to look at a flat panel display to gather patient information that is far away while also viewing virtual information (e.g., information regarding the surgery guidance) that is optically closer. In this case, the surgeon must change the depth of focus of the eye and visual attention to integrate information. This could potentially cause the surgeon to miss important information that could cause unexpected errors. This case is true for the VR environment when the surgeon performs training. However, this situation is also valid for other applications such as battlefield, maintenance, industry, and many others. Therefore, this talk will discuss research on perceptual phenomena in extended reality. First part of the talk will discuss three OST AR interface design issues: (1) context switching, where users must switch visual and cognitive attention between information sources, (2) focal distance switching, where users must accommodate (change the shape of the eye's lens) to see, in sharp focus, information at a new distance, and (3) transient focal blur, the focal blur user perceives while switching the focal distance. Then, this talk will discuss a perceptual image processing-based focus correction algorithm that develops a novel font for the AR system. We termed this "SharpView font," a font that looks sharper than standard fonts when seen out-of-focus. Finally, the talk will discuss the behavior of the human visual system with changes in perceptual depth in the real world, AR, and VR with eye tracking. Bio: Dr. Mohammed Safayet Arefin is an Assistant Professor in the Department of Computer Science at Colorado State University (CSU). Before joining CSU, Dr. Arefin was a Postdoctoral Fellow at the DEVCOM US Army Research Laboratory West (ARL West). Dr. Arefin achieved his Ph.D. and MS in Computer Science from Mississippi State University, USA. His research has been broad-based, centering on the topics of augmented reality, applied and visual perception, perceptual imaging, virtual reality, eye tracking, and human-computer interaction. Dr. Arefin won the 'Certificate of Commendation' from the SES Executive Deputy to the Commanding General of the US Army Futures Command and the ‘Director's Commendation Award’ in recognition of outstanding research achievement. Dr. Arefin has been active in the premier augmented reality and virtual reality conferences. For the seventh consecutive term, he has served as a Publication Co-chair in the IEEE Virtual reality (VR) and IEEE International Symposium on Mixed and Augmented Reality (ISMAR) conference committees. In addition, Dr. Arefin is co-founder and co-organizer of the Workshop on Replication in Extended Reality (WoRXR). |
|
|
|
Computer Science Department Colloquium Innovation in Interaction: The Future of Inclusive Robot Design Speaker: Dr. Kerstin Sophie Haring, Department of Computer Science, Ritchie School of Engineering and Computer Science, University of Denver When: 11:00AM ~ 11:50AM, Monday October 16, 2023 Abstract: Delve into the intriguing world of robot design, where we confront biases stemming from a narrow pool of designers. This talk is more than just an overview—it’s an invitation to broaden our horizons, embracing a diverse range of designers to achieve truly inclusive robot creations. We’ll also explore the Robot Theory of Mind’s role in enhancing the acceptance and effectiveness of interactive robots. As we navigate this design journey, we’re harnessing advanced evaluation techniques, blending machine learning with neuroscience insights from tools like fNIRS. Together, we’re refining and elevating the art and science of robot design. Bio: Dr. Kerstin S. Haring is an Assistant Professor of Computer Science at the University of Denver (DU). She directs the Humane Robot Technology Laboratory (HuRoT) and her research centers around enhancing Human-Machine interactions and collaborations. She evaluates trust, perception, and teaming with robot and machine systems and is interested in designing social robot systems. Before her appointment at DU, she was a leading researcher in Human-Machine-Teaming at the U.S. Air Force Academy. She completed her PhD in Human-Robot Interaction at the University of Tokyo in Japan, has a Masters in Computer Science and Cognitive Science from the University of Freiburg in Germany, and an MBA from the University of Denver. |
|
|
|
Computer Science Department Colloquium Deep learning in computational biology Speaker: Asa Ben Hur, Professor, Computer Science Department, Colorado State University When: 11:00AM ~ 11:50AM, Monday October 23, 2023 Abstract: Much in the same way deep learning has changed computer vision and natural language processing, it has transformed the way we analyze protein and genomic data. In this talk I will review ongoing research in my lab on the use of graph convolutional networks to analyze protein 3d structures, and our use of convolutional and transformer-based architectures for uncovering gene regulation. Bio: Asa Ben-Hur is a Professor of Computer Science at Colorado State University. Dr. Ben-Hur's research is in machine learning and its applications in computational biology. His work has been published in top venues in computational biology, biology, and machine learning. Support for his research has come from NSF, NIH, and DOE, totaling over $3M. Dr. Ben-Hur earned his PhD from the Technion, Israel Institute of Technology in 2001, has done postdocs at Stanford University and the University of Washington, and has been at Colorado State University since 2005. |
|
|
|
Computer Science Department Colloquium Using Social Network Analysis to Improve Team Performance Speaker: Jeni Cross, Professor, Department of Sociology, Colorado State University When: 11:00AM ~ 11:50AM, Monday October 30, 2023 Abstract: Science is in a period of change where larger and more transdisciplinary teams are forming to solve complex problems. Parallel to this shift in science is the emergence of a new field, the Science of Team Science (SciTS), which studies the social and organizational factors that contribute to team performance. I’ll discuss recent advances in SciTS, factors that predict team performance, and how social network analysis is being used to assess and improve team performance. Bio: Jeni Cross, Ph.D., is Professor in the Department of Sociology. She earned a bachelor s degree from Colorado State University and received her Ph.D. in Sociology from the University of California at Davis. She is a community sociologist, conducting research with and for community partners to solve community problems and improve quality of life. Her research has focused on issues of public health, place attachment, behavior change, community engagement, and using social networks to create systems change. One current research project, funded by the U.S. EPA, examines the impact of green schools on student and teacher performance and health. Her study, the Social Network of Integrative Design, is the first study to use social network analysis to describe the social processes of innovation in a variety of building design projects from community gardens to large commercial office buildings. Additionally, she is funded by the NIH Clinical and Translational Sciences Award to lead the Science of Team Science core at the Colorado Clinical and Translational Sciences Center. She is a sought-after speaker, regularly lecturing on cultivating transdisciplinary teams, developing behavior change programs, and creating systems change. Her TedX talk on the 3 Myths of Behavior Change has been viewed over 1 million times and been adapted in dozens of courses across the globe in social science an environmental science courses. |
|
|
|
Joint CS/ECE seminar Bringing privacy into the picture: new optimization goals for ML ⁄ AI in smart environments Speaker: Damla Turgut, Charles Millican Professor and Chair of Computer Science, University of Central Florida (UCF); Co-director AI Things Laboratory When: 3:00PM ~ 3:50PM, Wednesday November 1, 2023 Abstract: Smart assistive environments adapt to the needs and preferences of disabled or elderly users who need help with the activities of daily living. However, the needs and requests of users vary greatly, both due to personal preferences and type of disability. As handcrafting an environment is prohibitively expensive, in recent years significant research was done in systems that use machine learning to create a predictive model of the user. Machine learning, however, typically requires large amounts of data. A stand-alone smart environment, however, only has access to the data collected from its user since it was deployed. A possible solution is to perform centralized, cloud-based learning by pooling the training data collected from multiple users. However, uploading data collected from the personal habits of elderly and disabled users create significant security and privacy concerns. In this talk, we investigate the type of data sharing necessary for learning user models in smart environments and propose several novel considerations. We point out that data sharing is only ethical if the user derives a benefit from it. This implies that the decision to share data must be periodically revisited, it is not a commitment extending indefinitely in the future. We study the data sharing decisions made by users under several machine learning frameworks: local, cloud, and federated learning. We show that most users only benefit from data sharing for a limited interval after the deployment of the system. We also investigate machine learning techniques that predict whether the user will benefit from sharing the data before the data is shared. Bio: Damla Turgut is Charles Millican Professor and Chair of Computer Science at the University of Central Florida (UCF). She is the co-director of the AI Things Laboratory. She held visiting researcher positions at the University of Rome ``La Sapienza'', Imperial College of London, and KTH Royal Institute of Technology, Sweden. Her research interests include wireless ad hoc, sensor, underwater, vehicular, and social networks, edge ⁄ cloud computing, smart cities, smart grids, IoT-enabled healthcare and augmented reality, as well as considerations of privacy in the Internet of Things. Dr. Turgut serves on several editorial boards and program committees of prestigious ACM and IEEE journals and conferences. Her most recent honors include the NCWIT 2021 Mentoring Award for Undergraduate Research (MAUR), the UCF Research Incentive Award, and the UCF Women of Distinction Award. Since 2019, she serves as the N2Women Board Co-Chair where she co-leads the activities of the N2Women Board in supporting female researchers in the fields of networking and communications. She is an IEEE ComSoc Distinguished Lecturer, ACM Senior Member, IEEE Senior Member, and the Chair of the IEEE Technical Community on Computer Communications (TCCC). |
|
|
|
Computer Science Department Colloquium Learning to Measure the Perceived Location of Virtual AR Objects: An 11-Year Quest Speaker: J. Edward Swan II, Professor, Computer Science and Engineering, Mississippi State University When: 11:00AM ~ 11:50AM, Monday November 6, 2023 Abstract: In any use of Extended Reality (XR), an important aspect of virtual environment fidelity is being able to control the locations of virtual objects. Where are virtual objects located? How well can virtual objects be placed among real objects? Can a virtual object be co-located with a real object? Can a virtual object be located behind or beyond a real object (the x-ray vision condition)? Asking any of these questions requires answering the question: “How can we measure the perceived location of a virtual object?” In this talk, I will tell the story of my attempts to find an answer, in the form of an 11-year quest. Bio: Dr. J. Edward Swan II is a Professor of Computer Science and Engineering at Mississippi State University. He holds a B.S. (1989) degree in computer science from Auburn University and M.S. (1992) and Ph.D. (1997) degrees in computer science from Ohio State University, where he studied computer graphics and human-computer interaction. Before joining Mississippi State University in 2004, he spent seven years as a scientist at the Naval Research Laboratory in Washington, D.C. Dr. Swan’s research has centered on the topics of augmented and virtual reality, perception, data science, empirical methods, human-computer interaction, human factors, and visualization. Currently, he is studying the perception and technology required to give virtual objects definite spatial locations, including depth and layout perception and depth presentation methods. He is also studying efficient data science tools. His research has been funded by the National Science Foundation, the Department of Defense, the National Aeronautics and Space Administration, the Naval Research Laboratory, and the Office of Naval Research. Dr. Swan is a member of ACM, IEEE, and the IEEE Computer Society. He has served many roles in the technical communities of IEEE Virtual Reality (VR), IEEE International Symposium on Mixed and Augmented Reality (ISMAR), and IEEE Visualization. He is currently the chair of the IEEE VR steering committee. Previously, he served as one of the general chairs of VR 2021 and VR 2020, as well as a program chair for ISMAR 2017, ISMAR 2016, VR 2015, and VR 2014, and he was a member of the ISMAR steering committee. His service and scholarship were recently recognized by the 2023 VGTC Virtual Reality Service Award and the 2022 ISMAR Impact Paper Award. In 2017 and 2018, he served as Interim Department Head of Computer Science and Engineering at Mississippi State University. |
|
|
|
The Information Science and Technology Center Distinguished Lectures The Virtual Experience Research Accelerator (VERA) Speaker: Dr. Gregory Welch, Healthcare Simulation, University of Central Florida College of Nursing When: 11:00AM ~ 12:00PM, Monday November 13, 2023 Abstract: Virtual Reality researchers carrying out human subjects research typically find themselves working very hard, over considerable time, to run human subjects experiments that end up producing very narrow findings, with minimal statistical power, for a limited population. More than two years ago we realized the problem is with the fundamental approach we use to carry out human subject research. Over the ensuing 14 months, with input from hundreds of Virtual Reality (VR) researchers around the world, we developed comprehensive plans for a four-year effort to develop a transformative new research infrastructure in the form of a human-machine system (hardware, software, and people) that combines and extends aspects of distributed lab-based studies, online studies, research panels, and crowdsourcing, into a unified system for carrying out VR-based human subject research. The system will enable researchers to conduct user studies online, concurrently across a large, carefully curated, diverse, and dedicated standing pool of study participants. We call our proposed infrastructure the Virtual Experience Research Accelerator (VERA). I will share a bit of the story of VERA, the status, some plans, some newly discovered challenges, and some opportunities for community involvement. Bio: Gregory Welch is a Pegasus Professor and the AdventHealth Endowed Chair in Healthcare Simulation at the University of Central Florida College of Nursing. A computer scientist and engineer, he also has appointments in the College of Engineering and Computer Science and in the Institute for Simulation & Training. Welch earned his B.S. in Electrical Engineering Technology from Purdue University (Highest Distinction), and his M.S. and Ph.D. in Computer Science from the University of North Carolina at Chapel Hill (UNC). Previously, he was a research professor at UNC. He also worked at NASA’s Jet Propulsion Laboratory and at Northrop-Grumman’s Defense Systems Division. His research interests include human-computer interaction, human motion tracking, virtual and augmented reality, computer graphics and vision, and training related applications. He is a Fellow of the Institute of Electrical and Electronics Engineers (IEEE) and a Fellow of the National Academy of Inventors (NAI). His awards include induction into the IEEE Virtual Reality Academy in 2022, the IEEE Virtual Reality Technical Achievement Award in 2018 (VR 2018), and the Long Lasting Impact Paper Award at the 15th IEEE International Symposium on Mixed and Augmented Reality (ISMAR 2016). |
|
|
|
Computer Science Department Colloquium Universal User Data (UUD) for Virtual Experience Research Speaker: Dr. Gregory Welch, Healthcare Simulation, University of Central Florida College of Nursing When: 2:00PM ~ 3:00PM, Monday November 13, 2023 Abstract: Some key motivations for the development of the NSF-supported Virtual Experience Research Accelerator (VERA) are to mitigate the issues associated with laboratory-based human subjects research, including larger sample sizes, more diverse and inclusive samples, and faster turn-around. Despite more than a year of careful planning, and scrutiny of every aspect of VERA we were aware of, we did not foresee three major issues: in the distributed online paradigm of VERA each study participant will be using a different VR system, in a different place, and the differences in individual participant abilities could result in user data that is statistically different such that it might be excluded, when it otherwise should not be. These confounding three differences have motivated the development of algorithms, policies, and procedures for data signal processing and re-targeting to achieve Universal User Data (UUD) that can be appropriately compared within studies, across studies, over time, for as many purposes as we can imagine. I will share a bit of the story of UUD, including new found motivations, and some of our planned approaches. Bio: Gregory Welch is a Pegasus Professor and the AdventHealth Endowed Chair in Healthcare Simulation at the University of Central Florida College of Nursing. A computer scientist and engineer, he also has appointments in the College of Engineering and Computer Science and in the Institute for Simulation & Training. Welch earned his B.S. in Electrical Engineering Technology from Purdue University (Highest Distinction), and his M.S. and Ph.D. in Computer Science from the University of North Carolina at Chapel Hill (UNC). Previously, he was a research professor at UNC. He also worked at NASA’s Jet Propulsion Laboratory and at Northrop-Grumman’s Defense Systems Division. His research interests include human-computer interaction, human motion tracking, virtual and augmented reality, computer graphics and vision, and training related applications. He is a Fellow of the Institute of Electrical and Electronics Engineers (IEEE) and a Fellow of the National Academy of Inventors (NAI). His awards include induction into the IEEE Virtual Reality Academy in 2022, the IEEE Virtual Reality Technical Achievement Award in 2018 (VR 2018), and the Long Lasting Impact Paper Award at the 15th IEEE International Symposium on Mixed and Augmented Reality (ISMAR 2016). |
|
|
|
Computer Science Department Colloquium Tunneling between Optima (without the Quantum stuff): How linear lattices organize local optima for MAXSAT and the Traveling Salesman Problem Speaker: L. Darrell Whitley, Professor, Computer Science Department, Colorado State University When: 11:00AM ~ 11:50AM, Monday November 27, 2023 Abstract: This talk considers two combinatorial optimization problems: MAXSAT and the Traveling Salesman Problem, which have many real world applications. Given two “parent solutions” that are local optima, partition crossover can generate exponentially many new solutions that are guaranteed to be locally optimal in the smallest hyperplane subspace containing both parents. This allows partition crossover to directly “tunnel” between local optima, moving directly from local optimum to local optimum without using quantum methods. New theoretical results show that local optima are often arranged in multiple hierarchically organized hypercube lattice structures. These lattices can be exponentially large. For “easy” optimization problems, these lattices quickly lead to the global optimum. We also prove a new result, showing that simple linear equations can be used to evaluate subsets of local minima that are found in lattices. These results have already become state-of-the-art for heuristic Traveling Salesman Problem solvers, and have the potential to improve current MAXSAT solvers as well. Bio: Prof. Darrell Whitley has published more than 250 papers with more than 30,000 citations. Dr. Whitley’s H-index is 71. He introduced the first “steady state genetic algorithm” with rank based selection, published some of the earliest papers on neuroevolution, and has worked on dozens of real world optimization problems. He is a Fellow of the ACM recognized for his contributions to Evolutionary Computation, and was awarded the 2022 IEEE PIONEER Award in Evolutionary Computation. |
|
|
|
Computer Science Department Colloquium Is AI-Generated Code Really That Good? In Pursuit of Quality at GitHub Copilot Speaker: David Slater, Staff Machine Learning Engineer, Copilot Model Team, GitHub, Inc. When: 11:00AM ~ 11:50AM, Monday December 4, 2023 Abstract: GitHub Copilot, which recently hit one million paid subscribers, is the market leader in Large Language Model (LLM)-based code generation products. Maintaining this position amidst rapid innovation and fierce competition requires a rigorous and consistent focus on measuring, evaluating, and improving the quality of AI-generated code. In this talk, I will dive into technical challenges that we (Copilot Team) have faced in optimizing for code quality, as well as the experimental tools and techniques we have developed for solving them, including: (1) How we even measure, let alone rigorously evaluate, suggestion quality in AI-based code generation systems; (2) Limitations of current coding benchmarks; (3) The challenges of leveraging foundation models, such as ChatGPT, when the models and data used to train them are completely black box; (4) Tradeoffs between suggestion quality and other product-important metrics such as performance (latency, caching) and responsible AI (safety, security, copyright); and (5) The long tail of prompt engineering enhancements - most A ⁄ B experiments fail or are inconclusive while only a small handful provide significant quality benefits. I will conclude with a tour of some of those key advances and a conjecture or two about will (and will not) provide outsized quality improvements in Copilot for future AI-generated code. Bio: David Slater is a Staff Machine Learning Engineer at GitHub working on Copilot, focused on model quality improvements, GPU utilization optimization, and evaluation benchmarks. Prior to that, he was the Principal Investigator (PI) on DARPA GARD for Two Six Technologies, evaluating the robustness of machine learning models to adversarial attack. He was also PI for DARPA MUSE, applying machine learning to source code corpora, particularly for code summarization and labeling. His background is in applied security and machine learning, and has published in both areas. He has a Master's Degree in Electrical Engineering from the University of Washington. |
