🎤How is this Solutions Architect using ChatGPT for coding?

Taapsi 0:02

Hello and welcome everyone to another episode of Down and Dirty. And today I have with me Abhilasha Sinha. Abhilasha is a solutions architect at Walking Tree Technologies. She's been there for seven years. And before this, she's had an extensive career at Infosys, leaving as their tech architect after 12 years with the company. Abhilasha, it is so good to have you on the show. We are going to be talking today about Abhilash using ChatGPT but before we get into that, I would like Abhilasha, if you could just give us a quick introduction of what you do at Walking Tree and then we can dive deeper into the tool itself.

Abhilasha 0:38

Sure Taapsi. Thank you, it's a pleasure being here. So about myself, as you mentioned, I have worked on multiple technologies, like started off with Infosys as like a software developer and started with the basics. And then I've picked up multiple technologies, like starting from say Oracle package solutions moving into app development. And then finally, like in the recent two years, I've been more focused on generative AI. So currently I'm leading a couple of products we are developing using the generative AI technologies and overall the generative AI initiative. I'm also part of the presales team. So I get to interact with customers, understand the requirement, and design solutions for them. So, yeah, that's about my role.

Taapsi 1:30

Okay, you've piqued my interest. I want to know, generally speaking, because I know you can't get into specifics, what are the asks from customers when it comes to generative AI? What are some, are there some trends, some commonalities?

Abhilasha 1:46

Yeah. So like generally the content generation part is something that is common. And then one thing, I would say that I find the most useful, is when you're able to like use the customer domain-specific knowledge and augment ChatGPT. So we can do it like through some customization. So there are technologies like LangChain where you are able to connect ChatGPT to the customer knowledge base and then you can answer questions on top of that. So it's like see, ChatGPT when it interacts, it's more of the internet data, what we already have, what it is trained on. So this is like providing more knowledge to ChatGPT and enabling it to be even more powerful.

Taapsi 2:30

Understood. So where did your interest in generative AI come about like when and how did it come about?

Abhilasha 2:36

So it was like one of the earlier projects, I think two years back, we were working on a product which was like a question answering system on the USSEC, like filing all the data. So there we were using like models. So these were the older models like GPT 2 and Bard. So there obviously we had to do more hard work to train the models, and then we had GPT 3 coming in. So it looked powerful. And then when ChatGPT came in, it was like, wow, it really, I mean, it's like nothing… What else could you ask for? The human-like conversation, right? So yeah, now it it reduces the work in that aspect, you already have a very powerful model at your discretion to use it for your use case.

Taapsi 3:28

Understood, understood. So, before we get into the tool because I'm really curious about someone with more than 15 years of experience in this space and dabbling with similar, you know, generative AI technologies before even ChatGPT came about, but before we get into that, what does your life look like right now, as someone who's a solutions architect? From the time you wake up and kind of clock into work till your day ends, what's the spread like?

Abhilasha 3:55

So I mean, there are multiple you can say. There are some projects which I'm internally involved in. So, again, those are around generative AI. So we are using ChatGPT to generate code. That is one of our internal tools we are building. So I interact with the team and then plan for the day. I’ll help them if they are facing any issues then we have customer calls or demos. So getting ready for that, Then I'm also involved in some webinars. So we are actively promoting this technology also, in a way that is like trying out new things. It's such a fast-evolving thing, you know, like even in terms of technology. If I talk to you, the code also keeps changing. So I try out something, and then the next month I get to know there's a better thing that has come home. So it's secure to just keep up with the pace of the change. So yeah, I leave some part of the day to check out what is going on and then we have some webinars. So preparing for the planning, what should be the topics and things like that.

Taapsi 5:07

So, you're running the webinars.

Abhilasha 5:09

Yeah, we do quarterly webinars on generative AI. As a company, we do like monthly but for generative AI, we are focusing on maybe quarterly, depending on whatever new has come up.

Taapsi 5:24

Understood. And if you could share with me, what are some resources where you're doing research, you said, like you spend some time of the day learning about what's, you know, moving and shaking in this space. Can you name some of these platforms? Resources?

Abhilasha 5:41

So LangChain documentation is definitely one of the go-tos and then I keep checking YouTube videos that come up. Especially the recent ones on the topics of LangChain. So recently there's been like a new version, or I can say an enhanced LangChain Plus, which they call LangSmith. So when you are using it in your applications, you need to evaluate how well it is performing. So for that, LangSmith is used. And then in general, Twitter is a good platform, I follow these good you can say groups where we keep getting updates, and then there are some good Medium blogs also like Cobus Greyling is one whom I follow, he writes really good content.

Taapsi 6:34

Okay. Yeah, I can imagine. I can only imagine how fast it's moving. The conversations that I've been having with people who are dabbling in ChatGPT at different levels. I mean, the world of plugins [laughs], I can't keep track of like what all you can do by plugging into the technology. So let's jump into ChatGPT. So you mentioned that you first came across it, about two years ago because of a project you're working on. So now that we are in the era of ChatGPT and conversational AI, do you remember the first time you played with ChatGPT? What was that experience? Like, what did you ask it or what did you feel like? Just take me through that day if you remember.

Abhilasha 7:24

Yeah. So I usually post on LinkedIn. So when ChatGPT came up, I was like hearing people. So I thought, “I'll also do one post”. So I logged in and I think my first question was… my son's birthday was coming up. So, [I asked], “what are the gifts for a teenager? What are the good gifts I can give to him?” So, from there, I tried out variations on technology. So I asked, maybe with this technology, how can I learn it basically? So even if you want to learn something new, it gives good content as a beginner. And in pre-sales, there are times when you need to pick up something new quickly. So again, it becomes very handy to try out new things and learn new things. So yeah, that was something. And then another thing I tried, it was a friend's birthday. So I personally used to write poems, but now I'm like, depending on ChatGPT I'll give it a [bit of] content and it is really good for me.

Taapsi 8:34

What is a prompt that you would give it?

Abhilasha 8:36

So for our old friend who, who is very sincere and caring and whatever she does, she gives her best, like adding the attributes you want to and then it will write a poem for you.

Taapsi 8:51

Amazing. Okay, okay. So it's become your shayar…

Abhilasha 8:54

Yes, this has become very popular on our family group also. So now, on anybody's birthday, somebody has to come up with a poem.

Taapsi 9:03

And are you teaching people in your family on how to use and prompt ChatGPT, or is everyone just up to speed with it on their own?

Abhilasha 9:11

Yeah. So like they are from the IT background only. So they all, they all know it.

Taapsi 9:16

Okay. So that was your first introduction? So obviously, the more you interacted with it, I know that you've figured out the kind of prompts and how to prompt it better to get results that satisfy you, right? So let's talk a little bit about that. What would you say was your learning curve to get optimal results out of ChatGPT for your work?

Abhilasha 9:44

Okay. So I would give a technical example. So basically ChatGPT writes code, okay? Sometimes, we are lucky, and you'll get the result in one go. But mostly, I try it almost daily, if I'm trying out something new. So I'll ask ChatGPT like, “write a code for this”. but then there will be at least 2-3 iterations till it gives me a perfect code, which is working in most cases. Sometimes, because it has such huge data, right? So when it is planning the compilation, it sometimes picks up an old version of something that doesn't work now. So those kind of challenges come. But then again, when you're prompting, you try it out, you get some error, you give that error back to ChatGPT. “Okay, I'm getting this error”. So again, it will tell you, “apologies for the confusion, let me correct it” and then it will give you the code. So yeah, it's like working with your very efficient assistant, which is always available for you, right?

Taapsi 11:00

Right. So what does it do to your work ethos right now? If it is functioning as an efficient assistant, how much do you need an assistant? Do you need a real assistant? And to what extent? Yeah. And to what extent do you rely on, you know, would you need a human assistant versus ChatGPT as someone to help you do your work faster, right?

Abhilasha 11:26

Yeah. So it improves the speed of my work. Definitely. So like if I have some junior developers in my team, I can give them code to get started with a problem. So for them also, I'm giving a head start and a direction. So I mean, in the old world, I would have just told them, “okay, do this, do this, do this”, but now I'm able to give them a piece of work. I tell them, “it is generated by ChatGPT, so you have to do some work but this is the idea” and I mean the idea is like, you know, [in a] much more crystal format. So it's easy for them also to pick up from them. So that way, it's an assistant. If you understand the power, you can use it aptly, it's not like it can do everything, but then to some extent, it really helps and gives you that head start

Taapsi 12:16

Got it. And so how does the training or the working with the junior developer and actual developer work? So let's say you use ChatGPT to create code, right? And so there's a stronger, better starting point, a more structured starting point, yout give it to the developer. Now the developer is also using ChatGPT to find answers to figure it out to debug, right? So what are your thoughts on that if a developer, a junior developer is also relying on ChatGPT to make his or her life better or faster and then give you output that is inspired by, you know, artificial intelligence. What are your thoughts on that?

Abhilasha 12:56

So for a junior developer, I would say they should be less dependent on ChatGPT because they are still learning, right? So not totally dependent on ChatGPT, but then definitely like sometimes instead of going on a Google search and finding things, ChatGPT may give you the response way faster and better. Because I mean, Google again, you have to figure out which is the best thing. So that helps. But then, if you're just starting with the technology, depending on ChatGPT becomes as in you are not able to build your own skill, you become dependent. So once you are confident and you know the skill well, then you use it to improve your speed and efficiency, but don't be too dependent and like ensure that you know, the technology well, before you're making use of ChatGPT.

Taapsi 13:51

Got it, okay. So let's talk about use cases. Can you give me a few examples of how you, as someone leading the team… like what are ways in which you are leveraging ChatGPT? If you could just list it out, and then we'll do a share screen and you can take us through maybe one or two of those. So what are some use cases for you?

Abhilasha 14:11

So one use case is, if I have a problem statement, so I will generate the initial code or maybe even like try out differently. Like if there's a problem wherein I'm not sure which technology is the best, I'll ask ChatGPT to give me the different variations possible so that I am able to further apply it, like which is the best one, right? And then sometimes, if I have to write a blog or a technical post, I'll make use of ChatGPT to help me with the content. At least to some extent, I'll put in the words and then I'll try to ask ChatGPT to complete it, maybe write a post or write a small paragraph depending on the need. Then sometimes even when we have to think of the next webinar topic, we have some ideas and then we'll ask, it to maybe rephrase it into something more catchy or more nice. Then ChatGPT gives me a topic and even agenda points, and I'm able to plan. So just by taking the context, it gives like decent results. I would say then that for small modifications, we are able to use it.

Taapsi 15:32

You also mentioned presales, you mentioned a use case with presales. Can you give me an example of what that looks like?

Abhilasha 15:38

So with presales, like, suppose there's a customer from a domain which you're not much aware of. So, with ChatGPT you can in fact ask, suppose I have to apply generative AI to say, an ad agency. So I would ask ChatGPT for suggestions, “do you think we can apply ChatGPT or generative AI? Suggest to me the use cases”. So it gives a good list to start with and then you can definitely augment on top of it and add or remove whatever you feel is required or not. But that's why it gives a good head start to start with.

Taapsi 16:20

Okay. All right. So let's let's do a share screen. And if you can take us through an actual example of how you might prompt ChatGPT for any one of these use cases or any new one that we haven't yet talked about, that would be great.

Abhilasha 16:35

Maybe like if I give you an example for writing code. So suppose you have to try out whether through code, can I create an invite for Google Calendar, right? So you can see it gives you step-by-step all the things, what are the prerequisites, the entire code.

Taapsi 17:25

And what would you do with this? So now you have this, what's the next step that you would take?

Abhilasha 17:30

So if you see, it's giving me the Google calendar setup, maybe I don't have the details of what it is like, right? So I can ask for more details like, “can you elaborate on what is needed specifically?” So it will give you exactly all the details. But I can just ask somebody to try this out and see if it works. I mean mostly it will work in this scenario, I have tried this out. So this is something which works. But if something happens, where it uses all versions or things like that, then you just have to say, “okay, I'm getting this error” and it will give you the response. So yeah, this is one thing.

Taapsi 18:31

Got it. And how does something like this fit into your daily workflow? As in, at what point in your work would you need to put in a prompt like this, ask for clarifications and then ship it off to somebody else? Where does this fit in?

Abhilasha 18:49

So if there is a new problem statement, which I'm trying to solve, which we have not done before. It's more like you check the feasibility [of the solution] before giving it to somebody as a lead. You have to make sure that you're giving a good direction to the developer so that they are able to deliver. So that is where this becomes a good starting point for that.

Taapsi 19:11

Got it. Okay, okay. And any other example of something that you've, you know, possibly done in the last few days?

Abhilasha 19:29

So this was one example where I was trying to write a blog. So I was asking ChatGPT to suggest alternate titles. So this was something I came up with and then, and you can see ChatGPT came up with 10 other examples.

Taapsi 19:45

And what did you think? Did you use any of them?

Abhilasha 19:47

Yeah, yeah, I did use [them]. I think I used the first one.

Taapsi 19:50

[laughs] Okay, okay. All right. So, yes, you did mention that you use it to help you generate content, whether it's at the title level or it's at the post level. You use it with you know, as you said, feasibility-wise. What's an example of you know, a presales kind of prompt that you might put into ChatGPT?

Abhilasha 20:11

So sometimes… to improve the prompt in fact, we give a role to ChatGPT, like “you are a solutions architect who is part of presales and you have to come up with use cases there… for generative…” Yeah, I can say marketing. So you can see that ChatGPT is giving good ideas. So for your presentations, you can use this and then you can further add or delete content. If you want to elaborate on something, you can have a follow-up question, “give me more ideas on how you can use predictive analytics for the customer”. So again, it'll give you more if you want to elaborate on something. So it'll give you more use cases for the drill down.

Taapsi 21:38

Understood. And again, can you walk me through how this fits into your workflow Abhilasha? Like when does ChatGPT help with pre-, or how does it help with the pre-sales journey of yours?

Abhilasha 21:49

So, typically, suppose there is a marketing-related customer who is coming to us for our generative AI capabilities because we have worked on say, media, something similar. So we can do more research on what extra value we can bring to the customer. And that is where I'll do this research, then I'll maybe add to it further and that will form the base of my presentation with which I go to the customer pre-sales call. So that is how I will use this.

Taapsi 22:25

Got it. And you are part of the presales, you're there and you're in the call, you're the one presenting, is that it?

Abhilasha 22:31

Yes, I'm the one presenting.

Taapsi 22:33

You're the one presenting, okay. So I can see how this would be helpful in creating your slide topics even just to know what to put in front of.

Abhilasha

Yeah, yeah

Taapsi 22:40

Okay, okay. That makes sense. And can you tell me a little bit about how did you learn about [ChatGPT], did you take any webinars or courses or anything to learn about prompting? For example, when you use that example of saying “you are a solutions architect” and you know, that's a very specific way of talking to ChatGPT. How did you come about that kind of prompt?

Abhilasha 23:03

Yeah. So it's been more of like reading content. As I mentioned the resources, right? There is a lot of good content on prompt engineering and LangChain and the related technology. So I follow them regularly and that's how I have been able to pick up and apply the prompts. So like to start with, once you get the flow of it, it's more like conversing, right? And it's just that you're giving a role and you're asking the model to think. So that just adds a more specific context to the [prompt]. So like I mentioned, we are building this tool wherein the tool is again like an assistant to the developer. So whatever I'm doing here, we are actually trying to automate that process and using ChatGPT’s capabilities in generating the code. And that is where like of course, I mean, when you actually try to do… because as I said, when you are working, you don't get the result in one go. So to improve that you have to just keep polishing and adding more specifics to your prompts. And that's how like I am getting the feel of what to put.

Taapsi 24:24

And is that what you're also building, trying to think about how it can get more and more specific in its responses automatically?

Abhilasha 24:31

Yes, yes. So the idea is [based on] the customers’ user stories, like how the customer is going to use the application. So that is just text data, right? Using the text data, we are trying to generate code for the development, like when you're building the application. And also once your application is built, you have to test it. So there's some automation testing, right? We are even building the test scripts automatically with the help of ChatGPT.

Taapsi 25:04

Amazing. So you're not just using it to do your work, you're actually building on it.

Abhilasha 25:09

Yeah. Yeah.

Taapsi 25:10

Okay, okay. All right. Is there anything else that you wanna show here or should we stop the share screen?

Abhilasha 25:17

Yeah. So mostly these are the examples which I showed you.

Taapsi 25:22

Okay, okay.

Abhilasha 25:24

So there was one more example I can show you like, this is about an upcoming webinar. The webinar is about, when we deal with data, we have these dashboards and visualizations, right? Where you see the charts and graphs.

Taapsi 25:41

Yes.

Abhilasha 25:41

So we are trying to make use of ChatGPT to take the user input in plain text and build charts. So I was checking out different options of how we can do it. Like definitely there are multiple tools and we want to apply the right tool to build the ecosystem. And that is where I was actually taking ChatGPT’s help. So we use tools like Superset. Superset is an Apache tool which is used for visualizations. So again, we were trying to use ChatGPT to share ideas on how we can do it using Superset. So maybe I can show you that. So in fact, I started with Power BI. So how can I invoke dashboard generation in Power BI using APIs? So basically the idea would be using text, you convert it into an API code, and then using API, you're able to generate the dashboards in different tools. So yeah, this is another variation which we are actually applying in our solution.

Taapsi 26:51

Wow. And the entire flow would be automated from text to...

Abhilasha 26:58

Yes, exactly. So making it simple for the user, like you just ask in the chat and you get your business dashboard.

Taapsi 27:05

Wow. So okay, you got this response, and then what's the follow-up that you asked it?

Abhilasha 27:10

So Power BI is like a paid tool, Apache Superset is like the open source free version. So I wanted to try it out with something which is more available to everybody. So then using Superset, is it possible? Superset has APIs. So then I asked, “can you give an example of this API dashboard creation?” and then it gave me the content. So we are trying this out. And then you can see, like with the code, it also gives the explanation. So what you need to replace and what stands for…, like what variable is for what purpose. So it becomes very easy to understand. In fact, one more use case is that sometimes you get a code from somebody else and you're not able to understand, right? So you can just dump the code to ChatGPT and ask it to explain what this code is doing. So for developers, this is another useful way of making use of ChatGPT.

Taapsi 28:15

So were you happy with the response that you got by following this thread? What did it allow you to do next?

Abhilasha 28:23

It helped me to get started. We have started work on this and it is giving decent results. So, we have been able to achieve some part of the functionality.

Taapsi 28:37

Great, okay. Thank you for sharing that with me. It was great. So, yeah, we can stop our share screen. I have a general question for you. Other than the fact that ChatGPT, you know, at least a free version stops till September of 2021. What are some other limitations of ChatGPT in your line of work, if at all?

Abhilasha 29:02

Sure. So yeah, limitations, mainly I would say you cannot totally depend on ChatGPT for whatever code it is generating. You will need that manual intervention to try it, correct it. And then like, again, maybe ask a follow-up question if needed. So you have to be comfortable with whatever you're trying out if you're using it for code generation. Again, if you're using it for content generation, sometimes the data is from the internet. So, I mean, it's not unique. So you have to be careful about that part, that aspect as well. So, yeah.

Taapsi 29:45

Okay, okay. So, you know, I was stalking you on LinkedIn and I saw that you created this avatar… I couldn't understand what the avatar was saying. [laughs] I know it's supposed to be Hindi, but that was really funny, and I did watch it a few times trying to make sense of what it was saying. But you're actually stitching together many different solutions to see if… basically, if anyone who's, you know, who's probably not seen this on Abhilasha’s thread, it's your face with a different AI that's doing sound and a different AI that's doing image or rather like the image context.

Abhilasha

[laughs] The content, yeah.

Taapsi 30:24

Yes. And you stitch it all together. And the question was something around Java programming, right?

Abhilasha 30:30

Yeah. Yeah. So it was like… basically, it's for another internal project wherein we are building an avatar for an interviewer. And there we are doing the evaluation using ChatGPT. So the answer, the correctness of the answer and even scoring. So we are making use of ChatGPT to some extent there. So, yeah, that was like, I was initially trying it out. And somehow I don't know, there are these different tools we use. So that tool has the capability to add your own sound also. But then I was just trying out, so I picked up some Indian female voice and somehow it picked up Hindi language and it gave me an answer in Hindi. So then I had to make some corrections. But, yeah, the initial video came out so funny that I really couldn’t stop sharing it [laughs].

Taapsi 31:20

I know. But it's amazing that I've never seen this before, and I'm sure people are already doing this or trying it out. But this idea of putting your face onto a… I mean, for it to be an avatar, where you can ask it a question and go back and forth. That was amazing. I can only imagine where this is headed. Who knows? You know, our LinkedIn threads may start being people's avatars talking about doing a webinar, like you don't even need to do a webinar, it could just be your face. You know, your avatar.

Abhilasha 31:55

People will have… resumes will be their avatar talking about themselves, introducing themselves.

Taapsi 32:01

Wow, wow, unbelievable. Abhilasha, this was amazing. This was really insightful for me because I learned about the application of this as a work use case, not just someone who's doing it to get work done, you know, their own work done. Are there any last minute thoughts or comments that you have or are thinking about when it comes to generative AI, especially in the software development space?

Abhilasha 32:30

Yeah. So I would like to say, this advent of this model has made it very easy and approachable for anyone to use. Make use of this technology, understand its power, and then apply it your own use case. So definitely, we’ll see more and more adoption in the near future. Interest is building up and then everybody wants to really apply, because now they know that they can augment the knowledge with their own database. So they want to build a bot on their FAQ data or their company policies or whatever, right? So the the adoption is really going to go [up] in every field and domain, I feel. There is a bit of apprehension, with respect to data privacy and all. But again, definitely, like there are many other models also coming up and even OpenAI is promising data privacy. So, definitely, the adoption will increase and we'll see many more chat-like intelligent assistance. So I was seeing one recording on LinkedIn, where it was I think a sales call for Tesla. So it was a bot but the conversation was so seamless. I mean, that is how the future is headed. You would not know whom you are conversing with.

Taapsi 34:04

You know, I had this really good conversation with somebody once who was telling me that in this world where things are going to be generated using AI, we are going to put a premium on conversations with authentic users and authentic people, because you won't know who is authentic and not authentic at the other end of the call, unless you're actually sitting in front of a person, in front of someone having a real human to human conversation. And perhaps that's where the premium will end up being, because you can recreate anything pretty soon in the future with AI. So yeah. This was fantastic. Thank you so much for your time. I really appreciate it.

Abhilasha 34:44

Thank you Taapsi. It was really nice talking to you.

Taapsi 34:48

Same here. All right, have a good day. Bye Abhilasha.

Abhilasha 34:51

Bye bye.