#475 Laura Funderburk and Niels Brabandt on Building Production-Ready LLM Pipelines and Responsible AI Leadership
Laura Funderburk interviewed by Niels Brabandt on Building Production-Ready LLM Pipelines and Responsible AI Leadership
Artificial intelligence has rapidly moved from experimental novelty to strategic necessity. Yet one of the most persistent challenges facing decision-makers is not whether to invest in large language models, but how to implement them responsibly, effectively, and sustainably.
In a high-level interview with leadership expert Niels Brabandt, data scientist and author Laura Funderburk provides a rare executive-level perspective on building natural language and LLM pipelines that are ready for production rather than confined to prototypes.
Why most LLM initiatives fail after the prototype stage
One of the central insights Laura Funderburk shares with Niels Brabandt is that many organisations fail not because of the technology itself, but because they misunderstand the role of the large language model.
Too often, organisations treat the LLM as the centrepiece of the solution. Laura Funderburk argues that this is a fundamental mistake. Instead, the LLM should be treated as an add-on to a robust data science and software engineering foundation.
Production-grade LLM systems depend on disciplined data preparation, infrastructure, and engineering rigour. Without these fundamentals, even the most advanced models will fail to deliver sustainable business value.
The critical importance of data engineering and context engineering
Laura Funderburk emphasises that successful LLM systems are grounded in context. Context engineering, retrieval augmented generation, and structured data pipelines ensure that LLM outputs remain reliable, relevant, and aligned with business requirements.
This insight is particularly important for executives. Many leadership teams focus exclusively on model selection, while overlooking the infrastructure required to support it. Laura Funderburk makes clear that data preparation, storage, retrieval, and evaluation are the true drivers of reliability.
Understanding the true cost of LLM implementation
A major concern for decision-makers is the total cost of ownership. Laura Funderburk explains that the most significant hidden cost of LLM systems is inference, namely the cost associated with making repeated calls to cloud-hosted models.
Executives must make strategic decisions between hosting their own models or renting access through cloud providers. While cloud-based solutions offer lower upfront costs, long-term operational expenses may significantly exceed initial expectations.
This cost awareness is essential for responsible AI leadership.
Why evaluation, governance, and regulation cannot be ignored
Laura Funderburk highlights evaluation as one of the most neglected but essential components of LLM implementation. Without rigorous evaluation, organisations risk deploying systems that hallucinate, produce unreliable outputs, or violate regulatory requirements.
Governance, regulation, and data sovereignty must be integrated into the design process from the outset. This is particularly important in regulated industries such as finance, healthcare, and insurance.
Responsible leadership requires proactive governance rather than reactive damage control.
A practical 90-day roadmap from prototype to production
One of the most valuable contributions Laura Funderburk provides in her interview with Niels Brabandt is a practical 90-day roadmap for implementing LLM capabilities.
The first phase focuses on discovery, identifying the business problem, assessing available data, and aligning the initiative with strategic objectives.
The second phase focuses on building the infrastructure and pipelines necessary to ingest, process, and serve data.
The final phase focuses on deploying the LLM capability with user interfaces, evaluation, and governance mechanisms in place.
This structured approach ensures that LLM initiatives deliver measurable business value rather than remaining experimental exercises.
What executives must understand about AI leadership
The conversation between Laura Funderburk and Niels Brabandt highlights a fundamental shift in leadership. AI leadership is no longer about adopting technology for its own sake. It is about aligning technology with business strategy, governance, and long-term value creation.
Executives who understand the importance of infrastructure, data quality, evaluation, and cost management will successfully leverage LLM capabilities.
Those who do not risk investing heavily in systems that never reach production.
Conclusion: Leadership in the age of LLM requires discipline, not hype
The interview between Laura Funderburk and Niels Brabandt demonstrates that building production-ready LLM pipelines is not primarily a technological challenge. It is a leadership challenge.
Successful organisations will be those whose leaders combine technological understanding with strategic discipline, governance, and rigorous execution.
Artificial intelligence is not a shortcut. It is a strategic capability that must be built deliberately, responsibly, and intelligently.
Niels Brabandt
---
Mehr zu diesem Thema im dieswöchtigen Podcast und Videocast: mit Niels Brabandt: Videocast / Apple Podcasts / Spotify
Das Transkript zum Podcast und Videocast befindet sich unter diesem Artikel.
Ihnen ist exzellente Führungsarbeit wichtig?
Lassen Sie uns sprechen: NB@NB-Networks.com
Kontakt: Niels Brabandt on LinkedIn
Webseite: www.NB-Networks.biz
Niels Brabandt ist Experte für Nachhaltige Führung (Sustainable Leadership) mit mehr als 20 Jahren Erfahrungen in Praxis und Wissenschaft.
Niels Brabandt: Professionelles Training/Seminare/Workshops, Speaking/Vorträge, Coaching, Consulting/Beratung, Mentoring, Projekt- & Interim-Management. Event Host, MC, Moderator.
Podcast and Videocast Transcript
Niels Brabandt
AI is a big topic, and many people heard of AI. However, the question is: how do you actually build these natural language and LLM pipelines? And we have an expert on the matter here today, data scientist. Hello and welcome, Laura Funderberg.
Laura Funderburk
Thank you so much for having me, Neil. It's great to be here.
Niels Brabandt
Thank you very much for taking the time, and we're getting straight into it. So you wrote a brilliant new book. This is the new book that you just wrote, "Building Natural Language and LLM Pipelines." My first question, of course, is: what was your core motivation to actually get this book published? Because it's a highly developing topic, it's very demanding, it's very technical, it's not easy to write about. What was your core motivation to get the book written, get the book out there?
Laura Funderburk
Yes. So as everybody has seen within the industry, building LLM applications is a very challenging topic because, on the one hand, the technology advances very quickly, but at the same time, there's a very high demand for innovating with this technology. And so part of the motivation for me for writing this book was to provide the readers with the tools necessary to build LLM applications that are at the cutting edge of technology by using two key frameworks, Haystack and LangGraph, but at the same time allow the applications to remain stable and ready for production. So the key focus is production applications that use a large language model in the backend.
Niels Brabandt
Excellent. So your core motivation also is probably that, of course, the quality of the outcome should be better than what we see at the market right today.
Laura Funderburk
Exactly.
Niels Brabandt
Yeah. So without using too much jargon here, when people now read your book, what do they actually get out of that? Because probably we have some executives listening right now, and they say, "Look, I'm not a tech engineer, maybe. I'm probably not a tech expert. Is this still worth reading for me?" What's your take on that?
Laura Funderburk
Of course. Yes, it is. So the book is written in such a way that if you aren't a programmer, if you aren't a developer, you can still use the book to get up to speed with everything that's happened between 2023 up to today, get an idea of what you can build with these applications. And then there's a second layer of the book that is specifically targeted towards a developer.
Laura Funderburk
So the book is written in two parts. The first is, of course, the theory, where we get to break down the key concepts. We get to introduce the idea of prompt engineering, how it involves the context engineering. We get to introduce the topic of the cost analysis and the total cost of ownership. So the executive who might be more interested in, "How do I choose a model for the application?" or "How do I differentiate the key concepts that are important for my business?" That area of the book is covered within the theoretical part of the book.
Laura Funderburk
And then the second part of the book is tailored towards a developer via the use of Jupyter Notebooks and scripts within the repository.
Niels Brabandt
So basically, it's a book which can be used by both groups: the management, the executives, and the IT, the tech people, the developers by themselves.
Laura Funderburk
Yes, exactly.
Niels Brabandt
Brilliant. Of course, when I prepared for this interview, I asked people out there, "What are your main issues?" And one question came up very frequently where executives said, "Look, we tried this, but we always failed from the point from actually prototype to production with an LLM. We prototyped everything went well, and then we went into production and it didn't go too well from there." What are your opinions? What's your opinion? Why does this happen?
Laura Funderburk
I think the reason it fails is because we tend to rely excessively on the LLM itself as opposed to relying on software engineering principles and the rigor of data science. What makes this book different is that I'm arguing for reintroducing and using software engineering and data science rigor as the core foundation and treating the LLM as an add-on as opposed to making the LLM the main character of the story.
Niels Brabandt
And also with the RAGs, of course, just for the people who are not into tech, the retrieval augmented generation, that is basically making it more reliable, not only relying on what it knows. It actually gets out there and then finds the right solution then.
Laura Funderburk
So the focus when it comes when we cover the topic of RAG within the book, the focus is, of course, on the data engineering, preparing the data, cleaning the data, chunking the data, storing the data into the appropriate database, and then retrieving the data.
Laura Funderburk
But again, when we talk about RAG, the LLM is just the add-on. The LLM is the final step. Everything that makes the LLM grounded in truth is the data science rigor that is involved in preparing the data for consumption so that when we ask the question, the LLM receives exactly the information that we need.
Niels Brabandt
Excellent. Of course, one aspect which people often say, okay, they say AI, big thing, we need to invest here, but then they say there are often these risks of hidden costs. We have the initiative, the data, the evaluation, then the security aspect, then the change management around that. There are lots of hidden costs there. How can we avoid falling into that trap saying, "Oh, we're also fascinated about the idea. This has big momentum right now," and then falling for the trap of hidden costs afterwards?
Laura Funderburk
Yes. So the biggest cost when it comes to building with LLMs is inference. And the hidden cost is, of course, inference, namely making the calls to the LLM.
Laura Funderburk
The LLM is typically hosted on a cloud server, and so the hidden cost is either paying for the cloud provider to use their own LLM or us hosting our own LLM somewhere.
Laura Funderburk
The book addresses this via this idea, the concept of the total cost of ownership, where you pay more upfront if you host your own LLM. But then once that hosting cost is paid for, you get inference for free, minus the cost of electricity, of course, versus if you are choosing to rent your intelligence from somewhere else, say OpenAI or Assure, then you don't pay as much upfront, but the cost is higher over time as you keep making these renting prices. Yes.
Niels Brabandt
Excellent. Another aspect which came up when I was preparing is people often say, "Okay, I have to choose as a manager and executive between fine-tuning, then the RAGs we talked about, or simply a prompt-based approach. How do I make the best decision?" Because many managers and executives are very unsure because they feel they are not on the knowledge side of the game anymore, because what they basically know is how to use ChatGPT or any kind of other publicly available LLM. How can they make the right choice without putting themselves at risk and say, "I'm probably signing off costs here, which might cost me my career in two years' time"?
Laura Funderburk
Absolutely. So the biggest thing is within present or within modern time, the industry has made a shift towards RAG, prompt engineering, and specifically context engineering. The book argues very heavily towards context engineering.
Laura Funderburk
When it comes to choosing, I would say first focus on what kind of data you are dealing with. Is it the data that changes constantly? Is it data that requires specific knowledge? Is it data that you can put into a database once, forget about, and then—sorry, I'm losing track of track time.
Laura Funderburk
Is it data that you need to update regularly? If it's the kind of data that doesn't change often, you can pretty much focus on retrieval augmented generation, where the upfront cost is preparing that data once or a few times and then putting it into the database maybe with some maintenance.
Laura Funderburk
If it involves highly technical knowledge, if it involves specific jargon that the LLM is not necessarily trained to understand, then go for something like fine-tuning. And then prompt engineering and context engineering are going to be givens in both fine-tuning and RAG. Those are givens in both scenarios.
Niels Brabandt
Excellent. And then, of course, we need to go to what we call good evaluation. So what does good evaluation look like for LLM systems, and why do actually many teams say we'd rather avoid it?
Laura Funderburk
I'd say it's better not to skip evaluation. Evaluation is key because one of the key problems of working with LLMs is hallucinations. And when it comes to building applications with LLMs today, there's two flavors of LLM applications. There's RAG and there's agents.
Laura Funderburk
And in both RAG and agents, it's very important that we spend time evaluating RAG. The important thing is that the LLM is retrieving the right information and answering the right information. That's number one.
Laura Funderburk
With agents, it's very important to evaluate that the agent chooses the right tool for the job and then uses the right tool to answer the specific question. So I would say teams that are choosing not to evaluate are shooting themselves in the foot and creating a much bigger problem down the line.
Niels Brabandt
Yeah. And now, of course, they come up with immediately, "Let's talk about, let's say, insurance or banking, finance-related industry." And they say, "Look, we want to do all of that. However, we are in a regulated environment."
Niels Brabandt
And now they ask us, "What governments and documentation actually should we do before we deploy something?" So what do you recommend here? Because that, of course, is a huge area. What is the level of governance documentation you recommend before anything gets deployed?
Laura Funderburk
Before anything gets deployed, I would definitely investigate regulation and particularly who the customer or who the end user is going to be. So insurance for an application where the end customer is a medical patient is very different from insurance where the client is somebody looking for entertainment or media. And so I would definitely suggest getting familiar with regulations for who the end user is going to be.
Laura Funderburk
If choosing something like medical or legal, of course, the insurance premiums are insane because the liability is high. Oftentimes, when we're dealing with high liability, we need to be even more constricted about evaluation and about choosing where and how the LLM is hosted so that private information isn't sent somewhere that the client wouldn't want their data to be sent. So sovereignty, I think, is a very, very big topic when it comes to these applications.
Laura Funderburk
But when it comes to just addressing your question again, insurance, who is your target customer and what are the regulations for that target customer should be the two primary questions that should guide what LLM you're going to pick and how you're going to build your application.
Niels Brabandt
Excellent. Perfect answer.
Niels Brabandt
And of course, when now people come up and say, "Look, our decision-making process in our company is based on use cases, based on business cases," and then people say, "Well, good to LLM or RAGs, should we invest? Shouldn't we invest? Should we do something completely different with the money at the moment?" What is one realistic use case where LLMs, in your opinion, consistently deliver value today and probably one where they usually rather disappoint when it comes to results?
Laura Funderburk
Context, I think, is the area that they excel at. Their ability to produce information and use that information to produce rich answers is, I think, one of their strong suits. I am particularly fond of large reasoning models because in addition to having that contextual awareness, they also have the ability to reason.
Laura Funderburk
Large language models are very good at this context skill. Large reasoning models are particularly good at reasoning, and they tend to be very good at problem-solving. When we equip them with the tools that I introduced in the book, what you end up building are these autonomous agents that can solve highly complex problems while retaining reliability.
Laura Funderburk
So I think having the ability to use both, either for producing answers that are rich in context or using them for solving problems that are complex, both use case scenarios, I think, make them unique and separate them from different kinds of unsupervised learning models.
Niels Brabandt
Excellent. Very good. And one last question I have, because quite a number of CIOs reacted when I asked them about what is interesting for you for this interview, and many came up with if they'd ask you for a 90-day plan, not 900, 90 to 90, for a 90-day plan to get an LLM capability live but responsibly live, because they often get asked by executives who do not hold tech expertise, "Hey, we need something that we need to show to our shareholders, to our owners, to our investors." So what's your take on if any CIO asked you for a 90-day plan to get an LLM capability responsibly live, what would your answer be?
Laura Funderburk
So I would spend the first, I would say, the first quarter of those 90 days with what's known as discovery, namely what problem are we going to solve? We're not going to set up an LLM up in the air for 90 days just because it's cool or fancy. We always want to connect it to the business needs and the business problems that we're trying to solve. So number one, spend maybe the first quarter of that time identifying a business problem and then map out how the LLM is going to solve it. If the business problem is we want to have a model that can interactively answer questions about our knowledge base, that is going to be the business problem.
Laura Funderburk
So the first quarter of that time is going to be about discovery. What is our data? What is our problem? And then what is our infrastructure in place to solve our problem? From there, again, the first phase when it comes to implementing an LLM-based solution is the data. Data quality is the highest priority, making sure that you have the infrastructure necessary to prepare, ingest, and serve data. And then so that would be, I don't know, maybe the first month, discovery and data preparation.
Laura Funderburk
Implementation, of course, once involves setting up these pipelines. And the book I write talks about how to set up pipelines for ingestion and indexing. So setting up these pipelines, deploying the pipelines as microservices. The microservice itself is a REST API, so we can spend from day 30 to day 60 with the pipelines, deploying the pipelines as a REST API.
Laura Funderburk
And then the last part can be around the agent itself. Once you have the pipelines, a human can use the pipelines as a REST API to retrieve data and obtain information. The last 30 days can be about innovation with an LLM as an agent. Once the pipelines are deploying as a service, the last phase can be how do I power an agent to use the pipelines autonomously? And then, of course, the last 30 days should also be about the user interface.
Laura Funderburk
You don't want to give the user just a raw REST API. You want to give them a user interface. So you would incorporate a UX/UI team that develops a user interface that has these pipelines as the backend or that has the agent as the backend. So I would say that's my 90-day plan, from prototype all the way to production with a UI/UX that has REST APIs in the backend.
Niels Brabandt
And I think that was a really, really strong answer. So when now someone says, "Hey, I think Laura can really help us with this project," how can people get in touch with you?
Laura Funderburk
You can find me on LinkedIn at Laura Funderberg. I'm always happy to provide information and provide consulting information regarding how to set up these solutions. And I'm very happy to talk. I would say LinkedIn is the best way to reach me.
Niels Brabandt
Brilliant. Building natural language and LLM pipelines, build production-grade RAGs, tool contracts, and context engineering with Haystack and LangGraph. We've seen this book is not only brilliant, but also the person who wrote it is clearly, as is you, basically just demonstrated here today. So at the end of this podcast, there's only one thing left for me to say. Laura, thank you very much for your time.
Laura Funderburk
Thank you so much. It was great to be here. Thank you, Niels.