10x Charting the AI Frontier
10x Charting the AI Frontier
Accelerating Teams in the Age of Generative Intelligence
Historical Perspective: How has your team's approach to software development changed with the advent of Generative AI?
Startup Mindset: Sundar Pitchai said AI will impact ‘every product of every company’. Every team/scrum/org is now a startup. What it means - incremental solutions are not going to be sufficient. New solution with AI will need iterations - starting with MVP, collecting feedback and iterating while the underlying technology is also maturing
More Experimentation: Most mature companies over time become risk averse and R&D department reduces to D - that is development. Gen AI has forced teams to refocus on R - that’s research and Experimentation that it usually involves
Enhanced Efficiency: With the advent of Generative AI, we have been able to substantially increase efficiency. Generative AI automates a significant portion of the development process, generating code, predicting potential issues, and testing solutions. This allows our team to focus more on creativity and innovation instead of repetitive technical tasks.
Shift in Roles: With AI capable of generating code, traditional roles have evolved. Instead of focusing solely on technical skills, team members increasingly focus on understanding business needs, teaching the AI, and interpreting its outputs to meet our objectives.
AI Transformation: Could you describe the pivotal moment when you realized the immense potential of Generative AI for your team?
I have been a pretty early adopter of ChatGPT, started using it from its very early days, and have been using LLMs pretty much every single day since then. I will share a couple of examples - one from the creative realm, and one from the product realm.
Creative: As a leader, I’m always looking for interesting ways to connect with the team, and communicate a shared vision. Over the Christmas break last year, with help from ChatGPT, I came up with a Rap song for my org - what we do, why we do what we do, and how we go about it - sort of a purpose. I have never written a poem before and while I had to edit heavily, the final result was simply amazing. Let me share a couple of verses here..
Verse 1
We’re here to make sure every contact is handled with precision
Let AI and automation take care of every decision
So our customers, partners, and agents have satisfaction
And have the security to trust our reliability and remediation
Verse 2
We’ll use bots to help converse with the customer’s needs
And provide them with personalized care at lightning speed
Our tech and agents will work together in tandem
To help ensure we’re always taking the best course of action
Product: I have always been both amazed and disappointed by the variety of chatbots we have across the industry. If you lift the hood, the technology behind it is quite amazing and complex. It’s a very specialized discipline requiring understanding of NLP, the customer, specific use case, design to name a few. It takes painstaking details to get it right. While I don’t want to claim that Gen AI has completely solved chatbots, it was amazing what I was able to accomplish within a couple of hours of prompt engineering. The chatbot could understand a variety of intents, give reasonable responses, and handle quite a few “curve balls”
Limitations and Skepticism: Where do you believe Generative AI falls short, and how do you communicate these limitations to your team and stakeholders?
Probabilistic: Machine Learning — and, more generally, Artificial Intelligence — works with models that are designed, trained, tuned, and evaluated with a probabilistic framework. Output of LLMs can vary a bit, or a whole lot. Extracting exactly what you want out of an LLM is not straightforward today - it’s a blend of art and science.
Oversold: Throughout its 60+ years of history, AI has always been oversold. While Generative AI is a step change in AI evolution, it’s not going to solve poverty and world hunger, certainly not overnight.
Work in Progress: With the launch of ChatGPT, OpenAI unveiled a technology that solves a wide variety of use cases, albeit with heavy “human in the loop” requirement at the moment. Models, infrastructures, platforms, security and reliability - everything - is work in progress. That’s not a very solid foundation to build on compared to other fields.
Combinations of these require careful and continuous communication with stakeholders by openly sharing challenges and nuances around the highly experimental nature of the development journey.
Tools and Infrastructure: What tools, platforms, or frameworks have been invaluable in accelerating AI integration in your projects? What do you think of tools like Copilot?
Access to best in class models is a critical starting point. Companies like Uber have over the years made heavy investments in AI and ML. Michelangelo is Uber’s ML Platform. We have XP platform, and several in-house platforms to address a wide variety of use cases. You can read about them on Uber’s Eng blog. At the same time, we have been very open to adopting new tools like Langchain, Llamaindex, Pinecone, etc. We innovate at the cutting edge by combining our top-tier in-house tools and infrastructure with industry advancements.
Githhub Copilot, as it is today, is a promise of what’s possible. I was at Google Next earlier this week where I had the opportunity to meet with their technical teams. I was very impressed with their plans to completely disrupt the development workflow by significantly accelerating ideas to production for a wide variety of developers. To ground us in the reality today, it’s highly effective in pockets and a bit cumbersome to downright unusable in other areas. Your mileage may vary significantly - but this is not meant to discourage you. Only by adopting, experimenting and providing feedback for reinforcement learning, would you be able to realize its full potential.
Collaboration Dynamics: How has Generative AI changed your team's collaboration between data scientists, software engineers, and product managers?
Enhanced Interdisciplinary Understanding: Generative AI has fostered a better understanding and appreciation of each other's work among data scientists, software engineers, and product managers. By visualizing the code and its implications, it has bridged the gap between technical and non-technical stakeholders.
Iterative and Inclusive Decision Making: Generative AI’s ability to rapidly create and improve prototypes has fostered an iterative decision-making process. It has encouraged frequent feedback, collaboration, and collective refinement of products, enriching the overall development process.
Increased Focus on Ethics & Bias: The presence of Generative AI has led all team members to put a stronger emphasis on the ethical use of AI, and the potential for bias in its outcomes. It's now very much a shared responsibility, leading to deep, meaningful discussions that help to shape our products and services.
Renewed Focus on Security & Privacy: Generative AI opens up new vectors for security breach and privacy violations, and some of the tools from the past are no longer sufficient to address threats of tomorrow. New approaches are being brainstormed across the industry and it’s great to see renewed focus on these important factors as we increase adoption of this technology.
Training Approaches: You emphasized the "learning by doing" approach to training. What kind of hackathons, prototypes or projects have teams at your company built, and what were the key learnings from these exercises?
While I won’t talk about specific prototypes that we built as part of the hackathon, I can share in terms of themes what I’m also observing across the industry…
Chatbot: One of the most obvious implementations is chatbot - for a wide variety of use cases, all the way ranging from helping the end customer to helping internal employees across a variety of disciplines - customer service, HR, marketing, developer community, to name a few.
Developer Productivity: Ranging from automated code-review, to summarization of various engineering artifacts, best of the engineering creativity is being unleashed to eliminate some of the repetitive and laborious tasks - so that developer community can focus on what they truly enjoy, dreaming up bold new ideas and putting them into production at lightning speed.
Key Learnings: Prompt Engineering is a Lie! But it’s also a lot of fun. For a wide majority of the use cases you can get to 70-80% solutioning with the right set of prompts and models (combined with right platform and infrastructure pieces). Last 20% is hard. That’s why there is a human in the loop, as a light touch or a heavy touch mechanism depending on the implications.
Learning Curve & Education on AI: Your post mentioned educating teams on AI. What methods have you found most effective? For software engineering teams just starting with AI, what's the typical learning curve, and what advice can you offer?
Mix of Theoretical & Hands-On Training: There are a ton of educational resources available online, and published daily. I run a weekly newsletter curating the most interesting ones across research, practical, industry announcements to first keep myself abreast of the latest innovations and also inform the teams. This combined with organized hackathons creates a perfect learning environment.
Collaboration: Setting collaborative programs where experienced AI experts guide newer team members helps bridge knowledge gaps. It’s a great driver to break down information silos within the org. This can be internal experts or collaborative industry partners and customers.
Learning Curve: Frankly there is just too much great content out there from Andew Ng and others that I would advocate for everyone to go through them and judge for themselves. Unlike the previous generation of AI, new age LLMs have significantly lowered the bar to entry and experimentation. Mastery will require expertise in several different areas - deep understanding of the problem you’re trying to solve, platforms and infrastructure that can help you deliver the value at scale and AI models. If you’re new to AI, there has never been a better time to start!
Advice: Be patient and persistent. AI is a vast and continually evolving field. Prioritize learning foundational concepts well over chasing the latest trends. Also, employ a problem-solving mindset, focusing on how AI can be used to solve real-world problems. Experiment a lot and learn from trial and error - it's an integral part of the learning process. Don't be afraid to seek help and learn from others.
AI's Creative Potential: Can you share an instance where Generative AI surprised your team regarding its creativity or output?
Earlier I shared how I was able to craft a rap song collaborating with ChatGPT about my team’s purpose. Let me share a couple more examples…
Content - We have had BERT (which is a previous generation of LLM) in production aiding sentence completion. Of course, the latest advances in LLMs blow it out of the water, making the previous state of the art technology look exceedingly plain.
Code Generation - Coding has been long considered a very human feat. The fact that you can, in real time, translate your ideas into something people can use, through code is nothing short of phenomenal. While it’s far from perfect, in a particular software development task, the AI remarkably performed complex coding tasks, creating a more efficient solution than our developers initially conceptualized. The surprise was not just in the act of completing the task, but more so in the optimized way the AI approached it.
Long-Term Vision: Where do you see the intersection of software engineering and Generative AI five or ten years from now?
Democratization of Software Development: I foresee an increasing democratization of software engineering facilitated by Generative AI. It will allow more people, even those lacking technical expertise, to contribute to and influence software development, leading to an explosion in creative, diverse applications.
AI-First Organizations: I could see roles starting to shift more towards training AI systems, interpreting their outputs, and integrating them effectively into business strategies. This means a blend of technical, strategic, and interpersonal skills will be increasingly valuable.
Enhanced Productivity: With more refined Generative AI systems, a substantial portion of the software development lifecycle could be automated, reducing time-to-market significantly while improving the quality and reliability of products.
Policy Regulation: As Generative AI becomes more commonplace in software engineering, we expect a robust ethical framework to emerge, addressing issues like data privacy, AI bias, and black-box transparency. Learning and enforcing these ethical considerations will likely become an integral part of the software development process.
AI Strategy Development: Your post outlines several considerations for creating an AI strategy. Can you discuss a real-life example from your company detailing the challenges faced and solutions adopted in implementing an AI strategy?
While I may not be able to talk about specific examples beyond what’s already revealed by Dara publicly - viz github copilot, and agent copilot use cases of conversation summarization, let me share a perspective on key challenges we faced as a team…
Goals: Because the field of AI is so vast, it was tempting to try to do too much at once. We countered this by prioritizing specific use cases where AI could add the most value, such as enhancing predictive analysis and automating repetitive tasks.
Data: Garbage in, Garbage out is painfully true in the realm of AI. Having access to and curating the golden dataset to inform the quality of our results has been a key factor in our ability to make steady progress.
Infrastructure: This is where I believe we had an edge due to years of investment in AI. But we also greatly benefited from adopting the latest tools like Langchain, Llamindex and vectordbs.
Stakeholder Buy-In: Initially, acquiring stakeholder buy-in was a challenge given the substantial upfront investment and uncertainty around ROI. We held regular meetings to update them on the progress and potential of our AI projects, gradually securing their trust and support.
Regular Updates and Monitoring: Implementing AI is not a one-and-done task. Keeping AI products efficient and relevant requires constant monitoring and regular updates. We assigned dedicated resources to track the performance of our AI initiatives, investigate any issues, and recommend necessary improvements.
Each of these lessons greatly informed our AI strategy, allowing us to effectively navigate the challenges and reap the benefits of AI implementation.
Leadership's Role in AI Integration: You mentioned the critical role of leaders in driving AI adoption and innovation. How do you personally stay updated with the fast-paced advancements in AI, and how do you translate that knowledge into actionable insights for your team?
Active Learning: I personally make sure to spend dedicated time each week learning about the latest developments in the field. This may involve reading research papers, attending webinars, participating in online forums, or taking courses provided by platforms such as Coursera. As leaders, we need to create a culture of learning.
Networking and Collaboration: Networking with other AI professionals, participating in industry conferences, and engaging in collaborative projects often provide insights into the practical applications of new AI advancements.
Implementing Pilot Projects: As a leader, I believe in the "learning by doing" approach. I encourage my team to experiment with new AI technologies through pilot projects. This helps translate theoretical knowledge into practical insights.
AI First Strategy: I ensure that the knowledge gained from these activities is reflected in our strategy. Updated principles, adjusted goals, new project ideas, or workflow adjustments are part of this ongoing process of keeping up with fast-paced advancements in AI.
Skills of the Future: As AI takes a central role, what skills or qualities are becoming more valuable for software engineers?
AI and ML Understanding: As AI becomes more integrated with software engineering, a foundational understanding of AI and ML principles is becoming critical. This includes the ability to leverage, finetune and update AI models effectively.
Data Literacy: The ability to work with and understand data is crucial. This goes beyond just interpreting data and extends to data cleaning, processing, and effective communication of data-driven insights.
Adaptability and Continuous Learning: Given the swiftly evolving nature of AI, the ability to quickly adapt and continually learn about new technologies and methodologies is vital.
Collaborative Skills: As AI brings together professionals from a variety of fields including data scientists, software engineers, and product managers, strong interpersonal and collaborative skills are becoming more important.
Ethical Awareness: With increasing concerns about the impact of AI on privacy, fairness, and security, understanding the ethical implications of AI and incorporating this understanding into development decisions is of utmost importance.
Failures and Learning: Are there any significant failures or setbacks you've encountered while integrating AI, and how did they inform your future decisions?
High Expectations Vs Reality: In our early stages of AI integration, we underestimated the time and effort needed to effectively implement the new technology, leading to unrealistically high expectations and subsequent surprises.
One tricky issue we faced was straddling the line on token budget - like it’s really a goldilocks zone kind of thing - too little content and the bot makes things up. The more you can cram in there and it ignores important stuff - you’ve got to give it enough data to formulate an answer, but not so much that it gets confused.
Prompts that include conditional instructions are unlikely to be reliable today.
Hallucination is all too real - and can completely botch your use case.
Underestimating Human Alignment: In one of our early applications, while the technology behind the product was robust, we failed to take into consideration the notion of human alignment, which negatively impacted its adoption. E.g. Even if the AI solution is accurate, would you find it useful? The answer to this question, it turns out, is not very obvious. Deliberate effort is required to drive this alignment, use case by use case. The general problem of alignment is an active research area.
Future of Jobs: There's always a fear of automation taking away jobs. How do you see the role of a software engineer evolving, rather than being replaced, in this AI-dominated landscape?
AI as a Tool, not a Replacement: Automation is definitely reshaping the software development process, but we see it as a tool to augment the work of engineers, not replace them. While AI can automate repetitive tasks, the strategic and creative aspects of engineering will always need human input.
Teaching and Guiding AI Systems: Software engineers can expect their role to evolve to include a significant amount of teaching and guiding AI systems, as AI must be trained and supervised to deliver maximum benefits.
More Strategic Involvement: With routine tasks automated, software engineers will have an opportunity to be more involved in strategic, big-picture decisions about the software’s design, utility, and alignment with business goals.
Focus on Soft Skills: In an AI-dominated work environment, soft skills like communication, teamwork, and problem-solving-skills that can't be reproduced by machines-become more critical. Being able to translate complex AI functions into understandable terms for stakeholders and customers will be an invaluable skill.
Closing Thoughts: If you could leave software engineers and managers with one piece of wisdom or a thought-provoking question about the future of their craft in an AI-driven world, what would it be?
How well are you preparing yourself and your team to adapt, innovate and thrive in an AI-powered ecosystem where manual and repetitive tasks are automated, creative problem-solving becomes a foremost prerequisite, and continuous learning is the only way to stay relevant? How are you going to transform into an AI-first organization?
~10xManager