From Data to Intelligent Agents: Building Trusted Enterprise AI with Microsoft AI Foundry with Shubhangi Goyal [MVP]

Hello everyone, welcome back to another episode of VM 365 FAM podcast. I'm your host, Mocropitas, and today we are diving into one of the hottest topics in the Microsoft ecosystem, Enterprise AI. Over the last two years, we all have been talking about ChatGPT, Co-Pilot, and Generative AI, but building AI that actually creates business value in certain organizations, we cry out much more than great prompts. Today is guest understand exactly what happened behind the scene. Joining me is Shupag, Yiyal, Microsoft MVP, International, Rigenized Speaker, Zenyur, Daychallant, AI professional, and somewhat, we have spent more than indicate helping organizations turn data into intelligent decision making. We'll explore Microsoft AI Foundry, Azure, OpenAI, Data Science, Responsible AI, Enterprise Adoptions, the future of intelligent agents, and why good AI always start with good data. Yeah, Shupag, you're welcome to the NC65s as N, it's fantastic to have you here. Thank you, great to be here, and I'm looking forward to sharing all the insights that I have in this area. And yeah, of course, AI has really, I would say, full of load now in the past few years, and we are seeing literally being a part of everything we could potentially imagine. Yeah, yeah, first thing I congratulate you to your RENU and the MVP award. Can you tell us a little bit about you, how you get into the data world, data science world, and especially how you change to AI, and why Microsoft? Yeah, actually, that's quite interesting. So I think around 10 years ago, I would say at least more like 13 actually. So I pursued computer science engineering, and there was my first degree, and when I graduated from my course, I, at the time, like for example, data science was like one of the most, I would say, booming career arena. And when I joined my first job, which was back in India, I basically had the opportunity to work on some of the projects, which really sparked my interest in that area. And it led me to basically work on some of the projects there, which were mostly around like, you know, building machine learning models, and like, you know, analyzing data and seeing how predictive modeling works. So that's how I went into the world of, you know, data. And it became something which is like, I really enjoy working with. I can see like how, if you use effectively, that as a part of your, like, you know, as a business, how it can create impact, and how can find opportunities. And then like about seven years ago, I moved to the UK, and I graduated from University of Pat, I pursued my master's there. So I sort of moved from, I would say, a share to Europe in a way. And then my first job, I worked for an AI startup. And they are basically from data science, I entered into the arena of AI, and the rest is, as I said, like history. And I think one of the interesting parts of working in that job was like, I learned a lot more about AI, what AI really means. And I think when I joined that role, I thought that, you know, this is something just going to happen in maybe some years of time, but actually happened a lot before. And I was working on some of the projects there, which were already in the space, like for example, for example, wise agents, or like, building web char agents and so on. And then when AI came, it actually expanded that a lot more. And I just ended up like where I am. And yeah, like I have basically worked a lot of Microsoft tech in the Microsoft ecosystem. And also with Microsoft as like the customer in my first job. So it's like a lot of, I will say like using the ecosystem entirely, you know, working on Azure, for example, or even using data platform tooling and also like working now on Foundry. So yeah, this is how basically my journey began. I think it was more like, you know, it's just, I wouldn't say it's like I venture into AI. It's just like, you know, happened a longer way. And like, I enjoyed it so much that I just thought, you know, this is like something I would want to be a part of and remain a part of for as long as I want to be. So yeah, this is how it ended up. Yeah, yeah, yeah. You have said you have started at the start up. And now you are in the enterprise world. How different is working in these both worlds? So obviously like if we talk about working like big corporate enterprise world versus working in startups, the working cell is quite different. Startups are definitely higher-paced and like, you know, faster-paced, no matter where you work. And enterprise are like most sort of like the move, but like you have to be like, you know, they have to be like super like careful with the fact that you know, when we are moving, making changes, how does that impact like the entire organization? And then like when you work on start this, also you can have like access to a lot more, I would say cutting edge technologies, which I still have, but I think startups because they can move faster, they can experiment a lot faster than that's what I can say at least. In like big corporates, like you will have to make sure that, for example, even if you're performing a migration, then like, how does that migration impact the entire, for example, structure of an approach that you have built? How, how to change? And then what's the ultimate goal we are trying to check? So I would say enterprise AI has, you know, where if I see it like, evolved a lot more in the sense like you can see that previously people were experimenting with it, but now it has reached a stage of maturity with organizations, actively want to derive value from it. And having said that I know that there has been a lot of, you know, discussion going on that, you know, a lot of people don't see benefits and so on. But I think it starts with recognizing the right use case of where you can implement AI. Like are you trying to aim that, okay, I have this product I'm going to create or build with AI, but I'm expecting, for example, this ROI on this or I'm expecting that, for example, the effort of the process I'm currently using, it can be optimized or can be made better. So thinking about enterprise AI, it starts with that mindset that, okay, this is the use case and also be slightly realistic about it that at the end of the day, like, do we have in a free sources time and, you know, right people and, you know, the right platform to build it because there are robust designs really important when you build the system. It's not just about having a large language panel. It's also about having a robust design that actually, you know, caters to the need of like, whatever your goal is with that AI application that you are trying to build. Yeah, I say every company or everyone, everyone wants AI today, but how important is the data quality or was stayed with the data before I buy tools or edit? So data quality is a really, very big part of it. So today you design, like, for example, you build an AI agent or you go in the world of agent together, right, it still operates on some sort of data, enterprise data, customer data, whatever your organizational data is. It's not happening just that you can take some lush language model, plug it into your application, build like a front end and so on. It wouldn't work that way. And a lot of times, like, the data is not structured or it is not like the quality is not taken care of in a way that AI needs because, you know, it has hallucinates and there are times you might not get relevant outcomes or the right processes. And so it's super important to make sure that which people right now call this sort of context engineering, like, that's I think I would say it's a little bit like overlap there. So you have to make sure that the right context is provided to your AI application in the first place to make it like useful and also reliable for things. So today you are building an application that serves different arena of customers, maybe like there are students, they could be like people who are in mid-age or maybe like older population that you have to make sure that it is able to cater to this wide set of audience and, you know, is actually user friendly. I would say like user experience is great and then also like, big and trust it in the first place and that's where responsibility I ventures into the picture. And for most part, if you are a B2B company, right, like, the other business is going to purchase your product because they want to also understand how you have built the product firstly, then I think they would also want to know like how your AI has entered up like making some decisions, which I think is called sort of like an expandable AI like. So if you can't explain how it has worked, I don't think they would be able to trust it entirely. So it's super important these small aspects while building it. And then data quality is another key area. So for example, if I'm asking similar set of questions every single time or testing it, right, would it give me similar outcomes? Will it change? And then how much would that change be? And me as a customer, am I happy to have that change? I think those are some nuances which you can only know when you test it enough, but even like after building it. So I would say there is like a whole lot of process involved to ensure data quality is at the right point. And when we have data quality, what other things a company needs to be a, or a chief AI readiness? So AI readiness is a very broad term and there's so many things that fall under AI readiness. Like see, there are things for example, one side is like you are building things because you want to like make sure that your company is like, or organization is like using AI effectively. Then there is a second side where you also want to use it, which is like people call AI adoption sort of. And everyone is up to speed, they know for example, whatever the data stool is in market. So let's say if I'm using co-pilot, you should know how to use co-pilot for example, or actually strike brain prompts because sometimes you might have heard actually like people say, oh I gave this prompt, this is not the outcome I expected from it. And couple of sessions I think I've been on, like I asked a lot of people, you know, like out of curiosity, you know, how many people, how many of them can say they write brain prompts? And it's quite hilarious because every time I see one or two people just raising hands. So like even today, you know, because it's it's like three, four years now, like you would expect like people know this. But now what happens is like a lot of people when they talk about AI readiness, they don't talk or just building, they also talk about how the company as a whole is moving. Is everyone having access to the latest tools? How those tools are being used? How can you like, you know, make the productivity sort of better? So you might have heard about token maxing is like one of the things which have been happening, I think I did some articles around it like that people are trying to use that and make sure they're using maximum of tokens to show productivity. So there are a lot of a lot of things happening. And so when you talk about AI readiness, like firstly, it's if you're thinking if I'm builder's lens, you should be able to ensure you use the right language model, you have like a right use case for it. What is the goal you are trying to achieve with that use case? The right tool that fits into your stack and then obviously like good quality data that you can feedback into it and it doesn't have to be a knowledge base. It could be historical, some historical data, it could be some instructions or testing data, like whatever you can leverage essentially. And then second part of readiness is like making sure everybody in organization has access to that tools and no matter where they are in the career journey, they're able to use that effectively for their jobs. And you know because anything which is repeated can be automated. So if that thought process is there, you can save more time to do those tasks which require like a lot more effort because I know a lot of things I've been part of there's always some technical depth. And you know like there's always a thing you know, oh we have to do these things but we can't do this because we have to prioritize these X items. So maybe like that's one arena like people can think about when they think about air readiness. Okay this work is something we can probably automate with AI and let's see if it can do the job for you. Yeah. Would you say AI can compensate poor data? When you say composite, what did I do mean like like if we use AI for like like analyzing or checking anything that it's okay? Yeah or make the data better. I think a little bit about the co-pilot in fabrics so it can bring the data formats in the I don't know that's make sense. That's something like this. But did you see AI also can help here or is it really human manual work to get good data? So I think with data quality, I think like I would say like it can help you explore the avenues. But I still I feel like I know a lot of platforms offer some like facilities now in which you can assist you but I think you will still have to do some I would say at least a part of the work to make sure that the data quality is up to the level you would need it to be. So I think it's gonna be more like collaboration. It wouldn't be just one thing or the other but if you use it in the right way it can assist you but I think if I would see does like a human AI collaboration other than hey can AI do it, can human do it sort of and if they go hand in hand together they can be as a higher chance of making things useful especially in the space of data quality I would say. So I don't think it's just gonna be one or the other. I think it can help you identify a lot of avenues. It's more like a collaboration between the two things. I think I don't know before I data analytics was also a big big thing. How did you think the analytics part has changed from these times before before pre-area and as I first will now? So I think it's a very good question because I data analytics is obviously like it's data teams especially analytics is sit with the business and their job is to support the business and a lot of job actually involves building reports and analyzing large data sets maybe writing SQL queries to fetch data from whichever background you're using. Now when we talk about implementing AI it's like people are moving towards conversational AI where they want AI assisted analytics. So essentially what I've seen is like you still use you will still use the same data set but then like AI would be sort of like your assistant and that would facilitate the outcome for you. Now when we talk about it's a building dashboard right it can assist you for example hey I want to write that expression and maybe like somebody is not super pro in this or something so they can use AI to get some support. For example like you know they can use some ideas for designing visualization or like so it's more or more of like you know you can build it but it will assist you in a larger scheme of things but then like there's another side of it like for example you want to analyze data and you can build an AI agent but then for that you will need the data that you will still use for analysis and then you will have to verify that as well unless you have automated checks that can do that for you. So today let's see you're on an A/B testing or a campaign data and you've built an AI agent for like giving sort of recommendations or I would say build a report for yourself which you have to present somewhere. Now okay Dran it did the entire thing but then still you will have to check right like what's happening unless you have built some automated tests because I think what the larger part of whole process is like you cannot be sure unless there's some sort of validation checkpoint available that the outcome is right. So I see conversation analytics is one area which is definitely a lot in discussion and some people actively working on it and then obviously like dashboard building has become like you can see a lot of people using AI in conjunction with dashboards to like you know build them faster and I think like there are other arenas as well which I know that I know and I have been part of some where like people are actually thinking that maybe AI can do a lot more panellitic work because it can. But I think like for example designing by framing. So I know a lot of people use now cloud design for example because it can do that for you and previously we were using different other tools to build by our frame so it recreates a lot of things quite quickly and you can for example see them in a matter of five or two seconds or maybe less than a minute or so yeah like there are arenas you have to identify which arena you want to use it for and like you know maybe brainstorming is funny arena like when you are designing dashboards. So conversation analytics is definitely one thing which I think is a lot of people are actively working on to ensure that AI can support them and you know speed up the process of you know analyzing data. Yeah I think since I don't know 20 years a lot of people say excel is that it looks like it's still existing in a lot of companies but whereas really getting still I found it's what's happened with Power BI. It's so hyped and now nearly no one writes about Power BI. I mean I think I still I mean I'm using excel so in part so I wouldn't say it's gone so I think like it's different perspectives I would say like I mean like Power BI is a great tool and I've used myself like a lot of times in the past and like I don't know like what other people's opinion are but I think like with the capability that the co-pilot provides you can create like great things on Power BI like I've seen so many examples on you know socials and LinkedIn and so on people publish the reports and I think like with obviously with co-pilot becoming more you know more a little more like keep on developing and keep on becoming a bigger product I think it will probably go very well along with like how power way it will become like a bigger product probably from where it is now so I think it's like one of those arenas I think people have different opinions on this so I think like I personally feel like both of those products are like I've used them quite a lot and I still use both of them so yeah yeah you're also an expert in the AI Microsoft AI Foundry as a I foundry I foundry I don't know what what what name we choose I think everyone I once they use another name but what what is it exactly so Microsoft Foundry yeah it was before it was really I think it was basically Azure AI Foundry so Microsoft Foundry is sort of like your enterprise AI platform so if a developer yeah engineer or a data scientist you can use it to build like enterprise scale applications so it offers a lot of capabilities like for example that tool has a model catalog so you can see all the different models it offers and then it provides sort of like you know breakdown by cost throughput benchmarking and so on so you will be able to see like which model is performing the best or sort of like which is a top performing model then it you can build agents guardrails memory and you can also find junior models you will definitely need like to deploy one of the models before you can use it and then it also offers capabilities on like observability and tracing so basically is an entire sort of package I would say which has a lot of stuck to offer and like I would say like I've used like it a lot quite a lot and like it is obviously like new things have been added and like it's growing like constantly and I think like the product also if you use it like if you like try to build it you will be able to use it quite easily like it's very user friendly like in the sense like you can like easily get up to speed to like you know build stuff and you know test out and like new features are coming in every other day so basically if you are a developer or even if you are like somebody who is exploding the rena I would recommend you to give it a try because it actually helps you to learn a lot more about like you know how AI can actually be implemented and you can be your own agent you can wonder your own agent and you can also evaluate it so you have like this entire thing in one one place which I think is really cool yeah and and and how well there was Microsoft fabric fit into the story so like basically like when you have to create like a knowledge base which is sort of like the data that you provide or information that you provide so you can have fabric IQ so they have there's like fabric IQ there is work IQ and there is found break here so they these three are sort of like they can work in collaboration sort of and if you are using or building ontologies or semantic layer on fabric you can also use that in like conjunction with found reagents as well so basically like that's where like you can merge sort of those things together so they are independent they can be collaborative you can say that I think we have another tool co-pilot studio where did you see the the difference to AI foundry over co-pilot studio so like like both of those tools are great but like I think with foundry like I have found this like I feel like I like I use it a lot and I feel like you with co-pilot I would say both of them are great and both of them sort of like parallel capabilities but like using it for example building foundry IQ for me personally has been really easy and I would say really like useful and because a lot of my like building those applications involves like using data effectively and like in a larger sense so I think like and also understanding those models it gives me a lot more capabilities to use those things. What did you think is is AI foundry also for smaller mid-sized companies or is it an enterprise product? Okay one second give me one second. So I think like Microsoft boundaries obviously more enterprise AI and it basically helps to ensure that like it creates your gives you all the capabilities that you would need and like when we talk about like you know for example when you build the foundry IQ you can also implement Azure AI search in the backend or you can also implement for example things like you know work IQ and also like you can also create additional capabilities on that. So I think if you try building it like see there are two things is tools and then there is sort of like your knowledge base. So the tools section you can only upload files so it's quick and faster but if you're building enterprise agent then you will need sort of like more solid knowledge base and in those capabilities it's sort of important to ensure that like you have a larger set of data you can engineer the data effectively and make sure that like that data can be used within your application effectively. So no matter you use for example one or the other as long as you have like build the ontologies semantically effectively that should be able to plug in into the main tool so that essentially makes your job a lot more easier but yeah that is a process you know building that thing you will have to know what your data is and what you want to feed versus what you don't want to feed. And what kind of models are available and they have found? I would say there's a lot I think yesterday I saw you know probably over 180 or something models so it has basically from cloud to cloud opus, clouds on it it also has fable which was which basically came in co because of well all the node reasons it has grok and then like mistral and basically all the latest models all the GPD models it has 4.154O everything basically but I think like see the thing is like a lot of people think they have to use the latest model to build their applications but I think it's super important to understand that you don't necessarily need all the time latest model if you're going to find unit or if you're going to build like say a rag for it right so you have to make sure that what's a cost are you willing to pay for an application and decide then which one you want to use to build it and then the second thing you would want to also make sure is like what data will you use to find unit or build the rag application. I think that's important I know the example because I was saying rag is dead because of like larger context with was available but I think it's yeah I mean if unless you have a better approach then yeah you can you can debate about it. Yeah I think a lot of people now yeah they're using AI, using AI but we also have this Azure machine learning stuff and I don't know what to search makes sense especially for antiprizes to look where I spent I think expensive tokens and where can I use other things like that. Okay so what did you think how will the AI found really evolving over the next month? I mean there's I mean it's constantly evolving like obviously I think the new models will keep up from coming so that's gonna be always there for sure so I think like that's one thing which is going to be there then I think memory management because you know like the or agents cannot work effectively so I like that's something which probably will come along the way because like that will really I would say that's really the game changer as well in a way because if your AI does not remember anything about a previous conversation then I think it would probably be one of those things like it resets every single time so it does not adapt episode of so it requires to remember all those things that you're basically asking it previously so let's say today you are trying you're working in public sector you want to pay your council tax and maybe like you are a person and if your details are not provided it cannot give you the like account details or how to pay for example your tax or do you want to refund for that tax with super important to understand that that would be something which can actually very change it. I think it's a big part of also context engineering which was very at the end the term came way back in 2023-24ish start I would say but now it has just grown a lot and like it has become a super important part and I've also heard a lot about like harness engineering as well which is like something which people are discussing right now and I think like this this is one of the arena which I think definitely is going to grow as time goes by. And what role plays security and compliance inside the I foundry or is it inside the F I foundry? So obviously you can build guardrails so like I've seen people talking a lot about guardrails but like a lot of people they don't necessarily understand or know about what guardrails really mean so guardrails basically protect and make sure that like your agent does not like you know it doesn't have any unintended behavior right that's really the purpose of guardrails and so by default it foundry there are some guardrails in build but you can always equate any one so for example like if your application does not require any specific kind of content or protected material or something like sensitive data sort of so it helps to ensure that like okay this data this particular application should not bypass these things it should not we find a way to like tackle those things or for example you have fed some like the IID data which I mean most of us will not but like say hypothetically you did then how will like manage it so I think guardrails ensure that there is like you know those lanes that should be controlled and that your application does not perform any unintended behavior it is an option to also create redeeming so you can like create a redeem and sort of your test stress test your agent in a way that you can create scenarios and test it when it's gonna break one thing I always say is like you know we should perform like people should perform as much as negative testing when they build these things simply because what it does for you is like any situation that you might think can happen it should exactly behave in a way it should like it should not be saying oh this is yes but actually the answer is no for it so like it makes sure that you are actually active you're trying to test it without waiting for somebody else to tell you so imagine you build a product right you are a company and you go to a customer and they test it and you find those flaws for you then you they might not be convinced even if you fix those flaws effectively because now in the mind you're like oh I can't I don't know if this is gonna be like something I want to invest in so it's a proactive attitude also like check those things and just as many scenarios to make sure that okay I've done this my new diligence on this thing I know that these scenarios is gonna fail the scenarios gonna work so you already know all the pros and cons of your product and then let's say you know how to rectify them eventually because otherwise you will not identify them so yeah I think like this is one of the other interesting area where you can you have to practice like as it is possible AI where like you know just building something but also building something which is effective useful I think we spend a lot of I don't know money in AI and in tokens and in building AI also security and all all the other costs that are often done can is the CFO and ask yeah what is the the return of invest if they are or have you any tips how company can yeah elevate the every from AI investments that's a good question I think this is like a lot of people are discussing about just now as we I'm sure like somebody would have already little bit of like they imposed on the service fee I think yeah see air investment everybody wants use AI because they know that this is something which is gonna make like a jobs maybe like easier or like make us more productive and so on when we talk about using AI a lot of times people don't know where they want to use AI like which which this problem am I trying to solve with is AI as a tool available to me like this last language model because like you are a company which has a big customer interaction agent centers something like that and like probably your job is to make sure that the basic queries is supported by AI and then like maybe the very very complicated and once where is it users are asking then maybe I will do my agent it let's that's your case now you are trying to save the time but call me like how do you how that's basically where you have to see where you can solve the problem and how do you make sure that like AI is effectively useful because it could be possible that one of those cases is not the right use case for very one implement AI so I think it started identifying the right use case first thing and then understanding like what's my current cost that I'm paying for that same workflow or process versus what I think AI can do for me and then start being a little bit real about like comparison that okay for example I am paying X amount of pound million pounds here if I don't pay this I move this investment here will that give me RWA in the long term and it should also be thought of like you know not just like oh RWA is gonna come in like one month or two months maybe like have a plan of like in how much time do you think you can recover that money for example it's not like you know in one month you're gonna get the entire RWA right like it depends on like how big or small investment is in that sure and I think like then obviously like that from that like then starts like your for a building phase so in that start building phase you have to make as I said like a robust design which will cater to that investment that you are building and from there you have to start like being like okay now I can see that there is a possibility that this AI will let's say we 30% of my investment that I was doing in the process A versus process B so I think like there is like the thing is like RWA will come in at a point in time so I think in my understanding whether RWA will come is also the key it's not gonna be like in one week I'll get all the you know this value back or maybe in one month so that's thing and then identifying the right case I think these are the four or five things are super important when you are in organization trying to implement AI and then yeah also like also having a great like the right team because having the right people who are building the right architecture is important because you would know right like for example if you're a data platform architect right if you design the right architecture for example you are catering to five different products right in some like some areas the latency or the cost is more important in some cases real-time streaming data is important and you cannot have same thing or the other you have to think about each in every section so this is the same thing when we talk about AI like your architecture the design and then like whatever investment you're gonna make in that entire product or an application you are building is really the key to make sure what problem it will solve today you decide it's gonna be for sort of like encoding assistant for you or some sort of like I don't know like maybe assistant that will help you make DevOps tickets or like for example G-RT tickets or something then watch my current effort versus what's the effort now then what's the time I'm investing now what's the time after implementing AI how much money was going into this product before versus how much money do I plan to recover in one year three years time and this is how they should go about it rather than like expecting that this will happen just in one go and also it's an improvement process like you have to keep on improving the application a lot of times it's just sitting it's built in next year but like you know live language models they have to be monitored because sometimes they can change they can hallucinate even after you've done everything well so you have to keep them monitoring and tracing how they're performing and then if you think that at some point there is something else you can do and make it better because you know with context that you provide to the models right like you have to expand the old data out if you don't use it otherwise it's gonna be challenging for somebody to like check every single aspect of that application so I think there's so many parts of also building that product once you are entering that phase that the product team has to ensure to make sure that it performs in the way it is intended to perform and that's basically where it starts generating R-A because it's a whole cycle I would say from start to end. Yeah you say hallucination I think for me I think the I see the model is getting better better but now we are also in the world of AI agents where one agent said the other agent did this and so on so a lot of I'd say specialists I think that's experts I think it's come from mistrial the idea to build different experts but how how many chore can be monitored the outcomes that we have no bias or hallucinations in the work-restance. So like when you talk about hallucinations right like I know the latest model that we have gotten from like different all the different AI companies I would like I would say they are slightly better but obviously it's still there the hallucination is still there now people use temperature settings they have like different settings available that it can use to control the hallucinations but I think I would say like when you test your application there should be a baseline what's the acceptable behavior for my application and like what's the intended behavior so if my agent is going to hallucinate and give me deviation like for example with 5-10% then I would probably say it's not ready for production because I would not be able to accept that simply so I think like when you build applications especially with hallucinations it should be like different cases where you would have tested those hallucinations how much you are getting those hallucinations for then what's the baseline if it's like for example you know confidence code because after confidence code of your hallucination is like slight deviation which is probably would say like maybe like 0.0001 sort of or like very small which is not so much and also the answer is correct I think that's like one thing which you have to make sure you are checking every single time and then like maybe test a lot of times to see like what's the baseline you're going to get what's acceptable percentage you can also create like confidence scores for example if my application every time gives me 90% of confidence which I know is going to be right and then also automate some like checks in every step like not like in that conversation is in every turn like every time it interacts that's also I think very important and then like if it falls below that baseline then okay now we need to see so they should be like process for that so I think in that process you can make it semi-automate but then somebody will have to still check there should be some human in the loop especially if you're working for like industries which are very very you know sensitive like for example health care so you have to make sure somebody is checking those things constantly because yeah like you're I would say like like you cannot go wrong with like your outcomes I mean I'm like always like advocating for this like you cannot go wrong with what it creates we have to make sure that we're always checking those things step by step in every process so build evaluations check maybe like relevance accuracy how core and the result is is it having task adherence which means like it does that task it's a boost you do and then maybe give also some sort of feedback like sort of a feedback loop so that the limb tries to improve itself and like you know like also tries to find better ways to solve the same problem maybe look if you have given a better prompt it's structured properly I think these things sort of like help to understand where the hallucination is happening from and then how can we rectify it it's not just about the settings it's also about how you tackled outcomes once you get it. I think now we see I think every product now includes AI at home my my vacuum my my my coffee machine all has now AI in it and last of the in-depth that's a new new buzzword I see monitor this it's was responsible AI I don't know what it do but responsible I it's yeah it's as a new LinkedIn it's everywhere what do the actual mean in practice. Responsibility I responsibility I basically like you are using like the AI applications in a way that like you know way that it's actually fulfilling the task that you have assigned it to do and if it at some point like for example initially when chat you be decay I know a lot of people just fed in data or gave it information because they thought like you know just take all the information and that's it but you cannot put like any other data random data into you know applications because you know there's a large language models so using AI effectively is always important to ensure that like when you're getting a response you so if you are a user like for example if I'm the user right let's say I'm expecting a response in some capacity I would know that is this a response right or wrong I mean how are people how do people know that AI is getting wrong response if they're not the subject matter experts of their particular area right so understanding if I can rely on trust on this and on this outcome so like if you are generating some content for example you're using it from brainstorming do you think the large idea is going to be practical and like for example if you're asking on seeking information are those links reliable are the right links or that information is right so I think I recently tested one of the example like I was trying to build one agent for like you know just being my concert tax right and I was trying to see like how will it like for example take the data so it gave me some made a website you know in the start and I was like oh this link doesn't meet me anywhere because that's not the right website even and the information was incorrect so measuring and comparing that this is the outcome I'm getting this is the realistic outsmash should have been getting is making sure that you're responsibly using it especially like not just trusting whatever it's been given because even on both start ones is written that it makes an incorrect responses on the bottom of that the bar where you give the prompt so I think it starts with understanding that you have to like make sure you are checking everything before using it and like if it's not like sort of like an organizational application if you're especially using for example chatchipity or any other yeah I tool available in the market which this is a lot of them now and I think like with using a to for example you phone on a daily life first stuff so if you're building let's say you're planning a travel item right that's the one the muskony skills I've heard people using it for like making sure that like like it gives you the right option of the flight links and then checking those flight links actually available it basically starts from all those small steps and then obviously like from enterprise level you all know that like we need more testing and more like evaluation evaluations and LLM as a judge and you know making sure that all those things work hand in hand to make sure it's more responsible and then like for example if you're even if you're working in an organization if you find that some points giving wrong answers please share with other people as well so that they are more aware as well because sometimes maybe you identify a use case and other people haven't come across that so they will also be aware that oh like this gives what potentially happened like I couldn't get a wrong outcome for this thing so I think that starts with you know sharing more knowledge and then obviously like you know these like we are having for example community meetups as well so a lot of people come and share their projects and they share like things they have found so I think also listening to other people and like you know learning from them along the way also helps because for example they might have come across use case which you haven't so you were also know now okay maybe I will think about this next time and I'm building something new for me that I should be I might come across the same scenario and I will try to sort of not fall in that space where I have to do the like work to like you know reinterpret it so I think it starts with multiple practices to make sure that you are using AI responsibly yeah actually a lot of companies or selling AI governance frameworks there there are a lot of it did company really need an AI governance framework or it's okay to have IT in the data governance framework what's your opinion here I think oh AI governance should in like in the granting of things AI governance should be like big things since the beginning of the process because I think it's sometimes like we build the whole thing and then we think about governance I would say I think it should be in the start and it's actually one of the reasons for your AI being truly ready because like if you're like all the different processes that are important for example you work for a company which is super like sensitive, user sensitive data or legal or legal data for example NDs and so on so like it's super important that you have taken care of the governance because like it would be quite surprising if somebody is going to go and use it you know because like those kind of information and like you know you haven't taken care of the ethics and like you know make sure that all the sensitive data is tackled properly I think it's like a lot of importance there is on that so I think you you can use like you should make it a part of the architecture itself in the beginning of the process it should not be an afterthought that we do it once we have built a product it should be big in the beginning of the process and like I think like if you have a B2B or B2C company like like without governance I I'm not sure if like people will be like convinced you buy something because they have to pass the governance checks like ensure that the product is truly ready for like you know being used by the people so yeah I think like it's super important I mean framework wise like yeah I mean a lot of companies do sell a governance frameworks it depends on each organization if they actually use it I mean I would say like it's best to have it built in the start and to see like if actually that framework is going to be what file or like use before your product or the company that it works for and like if you have for example your own internal governance teams AI governance teams for example then they can support you in that process so it depends if you don't have internal governance teams you can support you then you might see external health but it totally depends on one company to another so I don't think one size would for all I think it depends from case to case cases those like smaller companies might I mean might not have a big governance team so they might need external support to do that but some companies do have a big governance team so they can support them in those processes yeah um what did you think about regulations I think we have here also two teams we have the the people they say it's allowing the innovation and the other saying it's an A-blig trust what what did you think uh I mean yeah I mean there's some industries where like uh it's important I mean it's sort of in a way also trying to ensure that whatever you are creating is the level as I said like entrustworthy so it's two sides of the same coin so it's all but I think like if your industry has this thing you know where they have to pass all these criteria then I think like you have to go through it and it's better to be like you know check all those things beforehand like and you know rather than like just putting something out like innovation is also go leave and like you know it's it's responsibly used and you know save to use also so I think yeah there's two sides of the same coin I would say so yeah I think some industries will definitely need to ensure that they pass all those barriers before they actually sort of reach a point where they are directly ready to be used by consumers and yeah this is awesome and how important that you think is it's actually yeah this is community in these AI area oh communities I mean there are lots now uh I mean communities are important and I think uh it's important to hear also like what other people are bringing like for example building because I think like collaboration and all the shittings are the best way to learn the new tech or any tech essentially and I think like a lot of places I have attended events or like I've learned a lot of things and the people I've created you know also like it also sparked my interest in the sense like you can discuss with those people what approaches they are followed so it does create a platform for people to upscale so for example somebody comes from core engineering like you know software engineering and then now they know about coding they know how to build software architecture as you see everything I listen so on but maybe like I want to understand more about AI which is more about like you know generative AI like large language models how they are built how the how they process information so now like obviously there are all of courses online but then like community meetups provide that sort of like a center spot where you can go and I generally say to people like you don't have to go to like you have to pick those events where you can learn the most because every event has their different things like some events like there's advanced AI some places they are more like topics which are like sort of like in middle like and some barely early start you know they have like topics which are very so you pick which one you are really trying to learn or maybe your job whichever you are in where it is more relevant and then you can go and attend those events and learn from them and then ask questions because curiosity is something which is super important especially in today's world to learn and understand more you should ask the questions if you have any confusion or doubt about something so that you learn along the way so I think community meetup are really really important in my opinion because they provide that free space to you know really be like learn from other people and I think either you are an attendee or you speak or like you're an organizer you will learn a lot because you will get different diverse perspectives and like if I'm a data scientist I will come from a data science lens or maybe somebody else who is a software engineer they will come from a software engineering lens and then somebody maybe is in analytics they will come from analytics lens everyone is using different tools and it's going to be everywhere so it's nice to know what's happening so for example in analytics we are seeing conversationally I being like one of the topics being discussed or in software engineering it's more like oh hey like now we can use AI to build code or make coding assistance sort of and if you are data scientist you would think for example oh like I use ML ops now maybe I'm going to use LLM ops or I use large language models to you know reproductive modeling how it can support there or if you're in finance or like marketing or sales how can I use AI for example will AI be able to help me create reports create campaigns analyze my campaign data so I think the different you know lenses so it's good to know all the lenses and learn how people are using it and if you have something to share you should definitely go and share that information as well. And Shavani you also lead a community the women in data are long then can you tell a little bit about this? Yeah so women in data it's a non-profit organization and it started back in 2015 in the United States and we have this chapter which is now I think almost 900 members so being a part of the community for a long time and I lead the chapter I think I organized 35 or 40 events so far in the past few years so the community and obviously just gone by 40-50% now so we have a lot of different women from the years you know walks of life and the organizer banks we helped them and dirt like fine rolls for example and we have a team chapters in click 57 countries or something now and obviously this is one of the chapters biggest chapters in Europe that one we have right now in Gijuke yeah it's a super collaborative fun place to be like anyone can join like there's like anybody like irrespective of is there like been on and they can like you know be a part of the community they can share their sessions they can also mentor other people they are also organizing a data thorn in the next couple of months so if somebody wants to you know build a team and participate I'm sure there's gonna be some AI topic that I think the topic is what's cooking this time so it's gonna be on some like data will be I think on cooking but then you create your own thing and there's bunny projects as well so somebody wants to you know team of it someone and build a project there's like that option as well so there's a lot of stuff it offers and yeah it's like it's not profit so yeah you can just join it for free and like you have reached the the value of like what it offers and I think you have also another community it's the build us foundry yeah so yeah bill is foundry I started this year and we already have like 150 plus members I mean we only done I think two or three events this year because we wanted to like build high quality events so we do we are building ones on one and the other and one other I'm doing this with another MVP so we are like doing and building this community together it's mostly focus on software engineering architecture AI and like you know startups because you know startups there's a lot more startups in London out than they were I think before especially because of AI so we are like looking for people who are like you know who want to share their real world expertise or knowledge I mean it's okay if something like didn't work that was just fine we just want to know like what's working what's not working and it's practitioner led community so we are looking people who are more like practitioners in their area and so we did like I think the launch event back in April and then we did one last month and then we have on one coming in September okay awesome yeah let's jump in the rapid fire round you say well the first thing that comes in your mind so coffee tea or energy drink during developed agents okay the third one energy drinks okay power be our our excel both pie Thor as KL oh that's just I think both because I use both of them so probably a year or fine tuning I mean it's case based so can't pick one or the other but like if I want to go for one I might if it's simpler I will go for fine tuning if it's not like a simple use case I might go with the other one yeah your biggest productivity hack focus fire bridge Microsoft product favorite Microsoft product oh there are so many I don't think I can choose if the Microsoft come come to you and say you get all the money and resources you need which feature will you develop I think like it's probably will be in front of you because I'm using that you know I will probably expand like the memory management feature like a lot more into a lot more capabilities yeah yeah then I come to my clothing questions the question what is the yeah the the yeah the biggest misunderstanding people have about AI like yeah like that it can solve like it can solve many problems but like it's not like it can it can't it needs to be in every everything like it needs to you're around if I use case for it to solve that problem for you so it will only help you you know where to use it effectively and also in the way you want to use it effectively yeah awesome thank you also for joining me today I really enjoyed this conversation because it showed up that successfully I isn't just a large language model or latest announcement it's about combinating trust data responsible governance solid engineering and the right business mindset so yeah thank you for for being here and yeah all all the listeners you find all the links in the show notes so you can connect this to Paggy and yeah if you enjoyed this episode so please leave a review and tell us your your colleagues and yeah thank you for listening and thank you so I for having you here. Thank you for having me. Yeah he was great chatting with you. Yeah good bye bye.