Webinar: Inspection Preparedness after TMF Transfer from your CRO

This webinar discusses inspection preparedness for a TMF inspection, with a particular focus on virtual and hybrid inspections. 

Key takeaways:

• Have a fit-for-purpose eTMF system that allows you to control the study narrative and is simple for the inspection to use & navigate
• Prepare your team in advance, having clear roles assigned, and ensure everyone is on the same page with regard to the trial story 
• Collaborate with your CRO prior to accepting a TMF Transfer. Ensure that you have the right process and technology in place to maintain an unbroken audit trail.
  
  

Full Transcript

Toban Zolman: Welcome to today's webinar. Marion, if you can advance slides, I'll do some introductions and logistics coordination and we'll get underway. Thanks everyone for joining today's Webinar. We'll be talking about working with inspections / inspectors and working through virtual inspections after TMF transfers.

We're going to focus on four areas today starting with how inspections have changed. There's been both some significant logistics and potentially even paradigm shifts post-COVID in terms of how inspections occur. We'll talk about system requirements and compliance needs to support those. Marion's going to walk through how to prepare for an inspection, and then we'll round out the conversation by talking about keys to be successful through all of that.

To start with some introductions, my name is Toban Zolman, CEO of Kivo. I'll be your software nerd for the day. speaking about how to use tools to support virtual inspections. But we'll start the conversation today with Marion Mays, Senior Vice President of Clinical. Marion has 30 years of experience supporting TMFs through a variety of perspectives as sponsor, organization, CRO, and software vendor. She'll do a great job outlining what's changed and how to support that using process and technology throughout a virtual inspection process. So without further ado, Marion, I'll turn it back to you. 

Marion Mays: Great. Thank you, Toban. My first question to everybody is, are you ready for an inspection? It takes a lot to get ready, and hopefully you started at the beginning of your study, getting ready for your ultimate inspection. So with that, I'd like to start with a poll, and with that poll, hoping to be able to get some insight into what you've seen, not seen.

I'm going to launch the poll, which is, "have you been through a recent inspection? Are you expecting one soon? And either way, how was it done?" Was it virtual remote, hybrid, or really just haven't been through one, so haven't had the experience yet. It looks like people are voting and answering our question, which is great. Most have seen the hybrid model. We're seeing some people talking about on-site, so they've actually had the inspectors come visit them. That's coming back in fashion as well, but usually there's some component of remote. So looks like quite a few people have voted. At least 60% of our participants have.

I'll go ahead and end the poll so we can get to the good stuff. Let me share with everybody what the results are. As you can see, some of them haven't been through an inspection yet, some are anticipating, so getting ready for that, and quite a few have seen some on-site. So that's great. Thank you, everybody, for participating with that. With that, let's kind of set the scenario a little bit of what I'm going to talk about.

You have a phase three study. It's closing. All of your sites are beginning to close down. Your CRO is quickly trying to upload all those documents and making sure that the study is complete. It may be missing a few documents, as we all see that happen along the way. So just trying to get everything together. And once it's all complete and the CRO says, "yes, we are saying that we've provided all the documents", you now, as the sponsor, need to take that on because inspections happen with the sponsor.

Even if your CRO may be keeping it, at some point in time, you're still going to end up with that study one way or the other. But let's say for this particular instance that it's being transferred back to you now; you need to accept it. So as part of that, you want to be able to be prepared, right? So are you ready? Is your team ready? Have you outlined your transfer process? Are you really ready to accept that in a controlled manner? And most of all, are your systems validated and able to support this inspection? And I think that's where Toban and I will talk a lot about how things have changed over time. So with that, let's talk a little bit about what's happened in the past, what we're seeing happening going forward.

It used to be we worked a lot in paper. We still may have some small component of paper somewhere. Let's hope that there's very little of that. But what we were seeing before was the differences between what's happening now is really around how things are accessed, looked at, and there's a lot more transparency going on now. So, inspections...before you would roll that cart into the inspection room, you'd have the files there available for them to access in a controlled way, right? You'd wait until they actually asked you for that file and you'd pull it off and hand it to them, let them read it and then put it back. Sometimes they'd ask you to make a copy of it or something like that.It was all very controlled, and somebody was copiously writing down all of this information as to what files they looked at, what questions they asked and everything. Some of that's still happening, right? Now we have an eTMF system which actually helps us control all of that, so that we know exactly what they look at, when they look at it. Even now, we can even see how much time they're spending on a particular file. Then we actually know what questions are coming at us based on the length of time and what they are looking at.

So a little bit easier to kind of navigate that, but they also have a higher expectation about that data so as to what should be available to them, what they should be able to look at, and also be able to connect the dots. So being able to put all that together, which from the sponsor side, loses a little bit of control over what they are seeing and how they're evaluating that information. So all important things and how things are changing along the way.

With that, what we have begun to see is there is the onisite versus remote inspections, and then we have the hybrid, which is a combination of these two. The standards haven't actually changed. Maybe some of the expectations have changed, but the actual standards of inspections have not changed.

They still want direct access to TMF information, and at least they need to understand your processes and everything around that. They still want to be able to print and copy and take some documents away with them to further evaluate. But the availability and retrievability still hasn't changed. Before, they would give you some time to actually go and find those documents. If they weren't readily available, they'd give maybe 24 hours. Sometimes they were more lenient and it would be 48 hours. Or you'd get the observation from the FDA and then you would have to reproduce those documents and show them that they were available at that time.So hopefully the dates all match up with that.

Now you have less of an opportunity around saying, "well, it's not easily accessible", or "I can't get that for you today". It's really hard to say that these days because it should be available. The expectation is that the system that you're using to manage your documentation makes it easily accessible because then they'll start to question around, "well, if I can't have access to it, how did your team have access to it? How did we know what happened during the study if people didn't have access to the information?" You start getting those types of questions. It's really important for you to think about the accessibility of these systems and being able to quickly grant access when you need to.

The other thing that happens is since they're self navigating a lot and you can't direct them down the right line of information, is there's an expectation that everything is labeled in a way that's easy for them to see. If not, there becomes that misunderstanding of what happened during the study, potentially.

If everything isn't connected well, the systems aren't accessible for them, the extra information is not accessible to them...they may start to question what's happened during the study, who made decisions, how did you make decisions. Because if they can't have access to the information, how did others have access to that information?

Let's talk a little bit about inspection timelines. Before, it used to be [the inspectors] would show up on site. They might be there a day or two, or if you were unlucky, they might be there over a week going through what was available to them, asking questions. Now things move a little bit faster. They also expect to be able to get information ahead of time. Granting them access to the information, whether it be from your QMS, your quality system, so that the processes and the procedures that were used can be reviewed before they come on-site in order to evaluate GCP expectations. Maybe they may want to review some of the data sets before they come on-site and then actually talk to the statisticians or the medical staff when they arrive.

They will have all their questions well outlined. They'll even ask for access to the TMF before they come on-site. So the expectations and the timing, their interaction with you may be less, which also leads back to that misunderstanding of information if it's not well organized and categorized, but the actual timing of the inspection actually gets extended. You actually have a longer time period where things are being evaluated and looked at because they're looking at all of that electronic information.

The real question comes down then, do you need an eTMF? Well, if you don't have one, how are you managing a big amount of data and information in a controlled way, meeting all the expectations around controls, access, and audit trails?  An eTMF actually helps support not just the regulatory compliance aspect, but allows you as an individual company to manage your data and know what's being done with that data. There is an expectation from the regulatory authorities that they can see what's been generated, that they can have direct access to the information and data, and that they can actually see what's happened with that data. What has that history been of that information? Who looked at those documents, who uploaded those documents, and potentially even who reviewed and approved those documents? That audit becomes a critical piece of the puzzle when information is being reviewed by the regulatory authorities. So if you have another system besides an eTMF that actually can manage and provide all of that, that's wonderful. But if not, you really do need an eTMF. As a small sponsor or a startup, you may not need one immediately when you're starting because your CRO will be supporting you.But in order to have good transparency and provide your documents in a compliant manner during your inspection and for years later, you need the appropriate software to be able to support you with that.

As we move through this process of getting ready for our inspection, I think it's important to know that there is certain system information that is required in order to be a compliant system. Things to consider when you're looking for an eTMF system to support you for your trials...

• Can the auditor inspector download or print documents?
• How much information do you want them to be able to have and access to have? Do you want them to ask you to be able to do that? Or do you want your system to be able to grant them that ability to do so?
• Do you use the reference model? So many inspectors now have become accustomed to using the Reference Model. It's become a standard. If you're aware, the TMF Reference Model that was an independent organization has now moved into CDISC, allowing it to be a more standard offering and more recognized standard across the world globally.
• Can the TMF index be created easily?
• Can you just print a report and know exactly what's in your eTMF? What documents are available?
• Have you been doing due diligence when you're setting it up?
• Does it allow you to track those milestones events and know when documents are due, maybe why they're not due yet?
• Does it also allow you to do cross references?
• Can you bring in other information and associate documents that maybe have been created elsewhere?
• Are you able to grant that access  easily from your system?
• How will you explain access limitations? Can you explain to the Inspector how things are controlled? Who has access, how they have access? Are you easily able to share that information and understand how that's all done?
• Records that may be embedded or links, can they be accessible? So if you have created a document and it's a link to the other documents, does it allow you to go across your...let's say you have a system that has multiple pieces to it, so your QMS and your eTMF and maybe just your standard EDMs or your safety system, can everything kind of work together easily?
• Can documents open in a timely manner? This is what we see a lot is a delay. So you click on the link to the document and how does it open and what does it take to open it? Are there times a day when the system is slow? So maybe planning that time or letting your inspector know know we're actually a European company, but when the US logs on, everything kind of slows down because we have everything sitting on our own servers, so it takes longer for that to happen.
• Do you have a naming convention that's easy enough for the inspector to be able to find things? Does it make sense? And easily able to navigate that structure, which goes back to the TMF reference model.

I'm going to pause in case there are any questions. Now I'm going to start getting into how do you prepare for inspections.

Now that we've got our system established and we know what's going on there, how do we start preparing for that inspection and being ready for that?  We did inspection readiness at the beginning, and we talked about that in some of our other webinars, so please go back and review those if you're interested. We talked about how at the beginning of your study, you're setting yourself up for readiness, so how you're going to manage all of that. But now we're actually going to have an inspection, so let's prepare for that inspection.

Do we know where all the information is sitting? Do we have a roadmap or an index or do we know what all systems are involved and where we can direct our inspector if we need to? Do we know what systems are generating that information? Also, if we've decided to leave something back with the CRO, do we know where it is at the CRO, who do we contact to get access to that, or do we have an easy access to that in some way? If this is a compound that we actually purchased and we've put this now into our system, do we know where the rest of the data is and is it sitting with the other sponsor? Just knowing all of this, how the data is flowing and where the documents and data are coming from. Maybe before you got your study from the CRO, they did a migration....Are you aware of that? And did all the components of that transfer back over to you when they actually sent your study? The legacy audit trail, did that come with it? Understanding where everything is and how everything works together is important.

With that, I want to bring up another poll for everybody and give you the opportunity to share your insight with us. Have you deployed an eTMF or have you thought about when is the right time to do so? Share your thoughts around, "Yes, we have an eTMF and we are ready for our inspection", or, "no, we don't really think that we need one". We're not sure because maybe we're just new to the company and we're not sure where the thought process is in that.

So looks like people are joining the poll and answering the questions.

Looks like quite a few of them have an eTMF, which is wonderful. So hopefully it is now more about getting yourself ready. I'm going to give it just a few more seconds and then we will share those results with you. Some are considering implementing an eTMF, which I think is good. If you haven't, we also talk about where are your documents? If your CRO is holding your information, where are you as the sponsor holding your eTMF or TMF documents that maybe should be in a controlled system? Looks like quite a few people, Toban, have an eTMF, but some are thinking about it and some know hasn't reached that level of where they're interested in actually having that.

Thank you, everybody, for participating. Let's move on with our questions here and getting ready.

Now that we've talked about where everything might be sitting with the documentation, let's talk more specifics around the eTMF system and things that you might need to consider in doing that. I'm talking a little bit about the implementation, since we have people who are starting that process and those who have already gone into it.

Make sure that you have qualified your vendor, understand the terms and conditions and what the ongoing support is, because on that day of inspection or while the inspector has asked for access, what if they have a problem? How are you going to help them? Who are you going to call? Who's your emergency contact at your vendor to get you the best help? Is there a help desk that you can access? About regulatory compliance...did they meet all of the marks on that? Are you comfortable with that? If you did single sign on, what's the documentation around that? We always talk about validation, so we want to make sure that we have all of that information and making sure that we can access it easily and readily in order to share it with the inspector should they ask for that.

They'll probably also ask you around change management, getting a system adoption. Who's using the system? How do they get access to the system? Where is training logged? Is there differences in the level of training based on the access? Yes or no or what's the difference and where's all your processes? Where's your SOP on using the system? Where's your SOP on managing trial master files? Do you have work instructions that go along with that? What do you have in place so that people know how to use the system itself? All of these things will be asked for or at least touched upon because, after all, you're going to have to also train your inspector.

In order to have a compliant system, you need to train people before they're granted access. Talking a little bit more about that compliance, did you follow your process? A lot of inspections are about did you do what you said you were going to do? Is it documented and is there evidence that goes along with that? That goes along with your eTMF system itself as well. Did you follow the process? Do you have an SOP for qualifying that vendor? If you do, did you follow that process? Is it documented? Can you show that evidence? Did you conduct an audit, whether it was a virtual audit or whether it was an onsite audit? Making sure that the system was validated, do you have access to that information? The new guidelines that have recently been published in May of last year actually talks about having readily access to the information on the system validation. You as the sponsor should have access to that information.

Let's talk about training. How did you deploy that training? Is it on just the system? Or did you also train people on your processes? Have you logged all of that? Do you have all of those things in place that show how to use the system? Do people have different roles and do their different roles have different permissions? Do they have different processes? How is that all managed? Who actually has the authority or the approval rights to grant people access? How are those accounts administered? If there are changes that happen, how do you take people out as well? Let's not forget that people have access for the time that they need it and then they should be disabled. Everything should be clearly available in the system in the audit trail as to still what they have done in the system. That should still be accessible. But what is your controls around granting access and taking access away, right?

What about a TMF? How are you managing your trial master file? Did you follow that process also?  Do you talk about how you use the system and manage the system in your trial master file TMF SOP? Everything needs to interlock and work together to be able to do that.

Let's say you chose to outsource your trial master file. How is that done? Do you have those SOPs? You might have outsourced the management of the trial to your CRO...Do you have that SOP? Can you easily access it? Can you walk through it and explain how things were managed? When you do that, you need to start talking about your oversight. What is your oversight SOP? If you don't have an oversight SOP, did you put that in your trial master file plan? These are all things that are important.

Now let's talk about risk mitigation, which is an important strategy that needs to be put in place when you're getting ready for your inspection. Have you done your inspection readiness? Did you do all those things and put those things in place in order to make sure that you were managing everything in a compliant way so that you're managing that risk? Have you gone through and done a TMF audit and maybe a system audit, so that you know all the pieces that go along with that and you understand how the system is used? Did you develop a plan and go back and make sure that you are addressing all of the deficiencies? Or how are you going to speak to them if they are acceptable risk to your organization? Have you put things in place that talk to that? What is your strategy around that? Did you document this process, which is a critical piece? Can you walk through it? Can you tell the story? Can you talk about how you identified the risk, how you determined that they were not really a risk, that they were not really something that you felt was critical or affected the outcome of your trial or patient safety or the overall compliance of your trial?

Now preparing for the inspection, are you putting all the things in place in order to have a successful inspection, which includes also your team preparedness? A lot of people haven't been through inspection, so they don't know what to expect. There's a lot of things that you can do. You can make sure that people know what their role is, not in the organization, but in the inspection process. Are they a runner? Are they going to be the one going and getting the information if it's not easily accessible to the inspector? Are they going to be somebody that might be interviewed and will be speaking to the processes or events or situations that happen during the trial? How about somebody who knows how to navigate that eTMF system or your safety system? That could be important if you can have a subject matter expert available to  quickly help the inspector navigate to where they need to be in order to see the documents that they want to look at, can mitigate a finding and, again, how to answer questions. You might want to consider doing mock interviews with the staff so that they can understand.

What I like to do is to put storyboards together. It outlines how are you going to tell your story and how you're going to talk about the events and situations that happened during the trial and where that information sits. Where does that documentation live within your eTMF system or other system that you will grant access to the inspector and talking about the CRO, so they may have managed it, do you know how they managed it? Can you talk to that?  The SOPs, the oversight information...where does that all live? Where you can show evidence of the activities and how they were done? Where are your processes? Did you follow those processes and time and contribution expectations to those involved in that? Can the other individuals that are being interviewed talk about that? Inspection prep is going through those processes and preparing that roadmap for everybody to use. Which leads me to that back room process.

We all have something that happens in the back room away from everybody else that's happening. Technology now is wonderful that actually helps us navigate that, because we can have a scribe in the room transcribing everything that's going on. The back room can easily see what's happening as the information is being typed and can prepare for that. 

People need to know how that all works and how that process is going to be managed so that not everybody jumps on the same question and tries to get answers and responses . Managing that response and how everything that happens...how do people get into the interviews, whether with technology, with the hybrid or the fully virtual? Do you have a team room that everybody's going into? Is there a zoom that everyone needs to log on when it's time for their interview? The other thing to remember is that you don't want everybody on that. How do they know when it's time for them to log in?Are you using another technology potentially to notify them that it's time for them to actually log on and join the conversation? These are all important things to sort out and organize in order to prepare for an inspection and be successful and respond appropriately.

With that, let's talk about the electronic system. We've talked a lot about how to prepare the team and your processes and everything, but let's do some system checks. Let's make sure that this is all working the way that we want it to. Especially now with the inspector having direct access, can we train them appropriately? Most of them only want about a 15-minute training. Can you get them up and running in 15 minutes in your system? Can the read-only and navigation...can you share all of that with them? All the documents are available to them so that they're not questioning why things are not filed there. So do you have empty placeholders? Do you have documents that are missing? Are you prepared to speak to all of that? What about the emails? We like to say "file, don't pile" your emails. There is functionality within the system to file the individual emails in the appropriate section. Don't make your poor inspector dig through a file that has 130,000 emails where they're going to have to find out what happened between the sites. The system should be available and accessible.

Make sure you check with your [software] vendor.We don't want anything going down during that time. Hopefully there's no updates or deployments during that time. Work with your vendor so that they know that [your inspection] is happening so they can either change the timing, delay, or make sure that your particular instance is not affected by anything and make sure that everything's validated and being maintained as it's supposed to. It's always good to check with them, make sure that nothing is happening that might be out of their own control. Even so, that coordination with your vendor, very good thing to do.

Let's talk about testing the setup. It's always good to test it. Let's see what [the inspectors] see, knowing what they're able to access, what they can't access, especially if you have a blinded and unblinded component of your study, it might be they should have access to all of it. So making sure that when you set up their account that they can have access to everything that they are supposed to. You might want to do some checks, make sure that nothing's going wrong and ensure everything works well. Sometimes we see that different screens have different fonts and different setups. If you're providing the laptop, that's great, you have more control, but this day and age, most of the time you don't. One inspector told me I only have my 13-inch laptop. So guess what? Make sure that everything I can look at is visible in that small screen, as that's how I'm going to be evaluating the information. 

Let's go to our last poll and hopefully we can glean some insight into this with everybody. Now that we've been through everything, are you ready for your inspection? Hopefully yes!

Looks like some people are absolutely ready to go, which is awesome. I love inspections. It's a time for us to reflect on the study and all the hard work that's gone into that. Yes, people are ready? No, they feel like they have some work to do. Some people are still working on closing their study, which is good. Now they can start working on prepping for that inspection as well.

It looks like 50% of our participants think they have lots of work to do...Toban, with that, I think it would be great if I hand over to you and you can tell them how to get ready for their inspection from a system standpoint after they get their study from their CRO.

Toban Zolman: Thanks, Marion. Let me take 30 seconds or so to distill some of the concepts that Marion talked about and how that influences how we think about the system.

A couple quick things about Kivo and about how I'm going to talk about stuff. My goal here isn't to show off Kivo, but really to leverage some of the features that we've built to help you understand, especially those of you that already have systems in place, how you could accomplish some of these concepts using your own tools.

I describe Kivo as a post-pandemic company. We took a product that had been market for years, but rearchitected that on the document management side post-COVID, which gave us the ability to align a lot of those features to folks working remotely in hybrid settings. Then when our customers came to us a couple of years ago and asked us to build a TMF solution, we really started as Marion advocates - at the end and worked our way back. Literally, Marion and I stood at a whiteboard and talked about this general process from the time of study end through virtual inspection and long term storage. The thing that struck me from a technology standpoint was, normally we think about technology and software as an enabler when it comes to inspections. But a lot of times, it actually opens up a can of worms.

When you look at some of the things that Marion talked about in her side of the presentation on getting ready for an inspection, in a traditional on-site inspection, you can march the inspector into a conference room, you can control their environments. You can spoon-feed them information.

When you give them access to your eTMF, you've got to create that same environment, but just the way in which you give them access exposes (as she was describing) your onboarding process, your SOPs, your training. There's a layer of complexity and risk, quite frankly, that doing virtual inspections starts to drive. So what I'm going to talk through is how we took that overall process and tried to mitigate risk at each of those steps from a technology standpoint. You can build a process, you can document that process, you can have SOPs and give an inspector access through the tech that shows them everything they need, moves faster than on-site, but doesn't open a can of worms and expose you to more risk.

This diagram that we've got on the screen right now represents our overall process that we follow at Kivo when we do a transfer and manage the process ultimately to long term storage. I'll spend a few minutes to give context to the inspection portion. If you've got the inspector accessing documents in your eTMF, and we can make an assumption that you have a CRO that's running the trial, they use their own system, whatever that was, and then at the end of that, it gets transferred back to you as a sponsor.

That transfer is super critical, and I won't spend a ton of time on that. Here we have an entire webinar that you can go watch on that. But what we've seen as an Achilles heel for a lot of our customers is the CRO basically just wants to dump that at the end of the trial once their contractual obligations are done, and get that off their plate. They may do that dumping back to the sponsor in a very uncontrolled way. FTP, thumb drive, you name it. So one of the things that we've really focused on thinking about that ultimate virtual inspection is ensuring that that transfer from the CRO to the sponsor is controlled. It's documented, it has SOPs, it has an audit trail, and there's never a time where it's exiting a validated system or a controlled process.

Paying attention to that is a critical piece to setting yourself up for success on the inspection side. Once you get access to that TMF and you have it in your possession, you need to get that imported very rapidly and in a controlled way into a validated, fit-for-purpose system. At Kivo, we've built an entire set of functionality to handle that migration. You need to QC that once it goes through, and be able to demonstrate that what is in the system that the inspector has access to has been through a process where you understand that it's reliable, that the audit trails are correct, that the versions are correct, that the inspector can see in that fit-for-purpose system when documents were approved, when they were effective, so that they can start to trace through the TMF and actually understand who had access to what documents, when, and all of that is controlled. Once that's been completed, then you need to be able to manage that TMF.

The thing that I think we've learned and seen with a lot of our customers is once they take possession of the TMF back from the CRO, there's still a long tail of documents, site closeouts, things of that sort that they have to be able to manage, get into the eTMF and ensure that all of that is there, consistent and available for the inspection. You go through the inspection, but once that is complete, you still have to retain that documentation, retain the eTMF for an extensive period of time. So having a system, or at least a process defined of where you're going to move it into a system to support that long term storage is super critical.

We're focused on the inspection today, but there's a lot of connective tissue in terms of process and technology around that that all has to link together for the inspector to get the right story, to not open a can of worms and question things because of how you have given them access to technology. How you have or haven't trained them, how you have or haven't presented SOPs...all of that stuff is a critical piece to the overall narrative that you're telling about the study, reassuring and reinforcing that the right process and practices were followed leading up to the inspection.

Marion, if you advance slides, the next piece that I really want to focus on with this is how you manage content and access for the Inspector.

Marion spoke about the eTMF reference model. We have found that that is a super critical piece to how you present your eTMF. In Kivo, whether we are letting customers build an eTMF directly in the platform or we're taking an eTMF from another vendor system and migrating it in, we really try and reinforce the TMF reference model. You can see in the screenshot, we've got these three columns that effectively let the inspector navigate their way through that reference model. We use the zone numbers from the reference model. You can search for those. It makes it very familiar to the inspector so that they can get to the content they need very quickly.

As part of all of that, you need to be able to manage missing content, safety information, revisions received, post-study, whether it's site, close out or anything else, and be able to incorporate that into (ideally) a single view for the inspector. You want them to be able to find and view any documentation, search across metadata fields, especially if you have a CRO that was layering in rich metadata. You want them to have access to all of that and then you want to be able to distribute those documents and data throughout your organization or to your partners and give your inspector focused access inside of that. What this ties into then is giving inspectors controlled access to that information.

The way that we handled this at Kivo, we literally built our tool around an inspector user role. It's something where you can give an inspector access to a trial in Kivo that's typically a cabinet. Once they have that access, they can see just approved documents and just placeholders. They don't see empty folders, they don't see areas where you don't have content. In other words, you are guiding their discovery of what documentation you have. We also highlight when documents have been approved and show when those approvals happened, especially if you have multiple approved versions of a document. That approach really gives us the ability to effectively produce a curated view of the study.

I'm careful with the term curated, though. You are not giving them a view of something else. They have access to the entire study. They're seeing real data, real documents, the actual eTMF, but they are getting a view of that that supports and accelerates their ability to navigate it, to make sense of what's there and to really give them a richer context for the documents that they have.

One of the ways that we do this is we leverage a concept called Placeholders, where if you don't have a document, you can create a placeholder for that document. Those placeholders can have dates on when the content is expected. So if you do have trailing documentation that you haven't received at the time of the inspection, you can put a placeholder in it. You can indicate when that content is expected and even why the content's going to arrive at that time.

The other scenario, though, is you can exclude placeholders in Kivo, and there are certainly ways you could accomplish this in other systems as well. But in Kivo, we let you exclude a placeholder and identify why it has been excluded so the inspector can see, and I've given three examples here, there's not a document available for that section of the eTMF, maybe because it's not relevant for the study. It's available with the CRO, or it's available in another system of record that you're maintaining as a sponsor that you're going to give them access to. There's a variety of ways to handle this.

Again, the goal really is to remove ambiguity for the inspector and to focus what they are looking at so that they aren't asking questions or second guessing what you've done based on how it's presented in the technology.

The other piece that we have really focused on at Kivo is intelligently linking documents across the platform. I've got an example here of a clinical study report protocol. This is how it shows up in an eTMF. But this particular document, as you can see here, is a document alias. It lives in a regulatory cabinet in our system, but dynamically shows up in the eTMF. This is a really effective way that we have found so that you are not adding risk into your process, because the inspector is seeing one set of documents. But if they start to poke into your document management system, they find out there are other versions that have been approved that were effective, that may have been communicated to some investigator sites, but aren't part of your eTMF. This is a way for our customers to mitigate that. They can intelligently link a document into an eTMF. The inspector can see it, they can see when that document was approved, which versions were approved, or in effect at different times. There's a host of ways, I guess I would say, to leverage the technology to reinforce your story rather than expose yourself to risk and have an inspector start to really question what's happening.

I wanted to conclude here with some tips. The first thing I would say is that you really want to pay attention to how your eTMF system enables inspectors to use navigation and search. What features are part of your platform to really support the inspection? Can you do remote usage by the inspector? We've had customers come to us late in the process, realizing that the way that they had archived an eTMF or were storing it on their local network gave them no way, no path to give an inspector access to it. They maybe had files sitting with exported audit trails, which made it very cumbersome for an inspector to make sense of the data. Some systems, when they export the eTMF data from the CRO, they flatten the hierarchy, so it's not visually presented in a way that's navigable like you would find in the reference model. So really, just thinking about how you're storing your eTMF and what that user experience is going to be for the inspector is super critical.

The next thing I would say is really understanding CRO study transfer. This is an area where I think a lot of smaller companies, especially or companies that haven't done a lot of clinical trials, struggle.

A lot of CROs oversimplify what that transfer is going to look like. Quite frankly, their incentives aren't aligned to the sponsor's incentives in terms of how that transfer is going to go, what their obligations are. If you haven't been through that before, you can really set yourself up for a lot of challenges. Understanding how your CRO is going to transfer that and making sure that you're handling that in a really robust and compliant way is super critical. This is an area where I would say, I don't care if you work with us, but get outside help. Get a perspective from someone who's been through that to make sure you don't set yourself up for inspection risk as part of that transfer.

Two final tips here on inspection readiness. One is on training. Prepare your team. Marion talked a lot about the back room and storyboarding. I'm a tech guy, but I got to tell you, from a process standpoint and making sure that stuff runs well, that's super critical. Having everyone aligned, telling the same story, and really making sure that there's a consistent message and a consistent presentation of that message both verbally and through the technology is really important.

How you train the inspector and having that map to match the training you give your team is also really important. It reinforces that you're following proper process, that everything you're doing is above the board, maps to your SOPs...it just streamlines the entire thing. It's really about controlling the narrative.

Then the final piece is ensuring that you have a validated and compliant system for storing your study content. I would add to this and say the term "fit-for-purpose" is used and probably abused a bit in the context of eTMFs, but it's a big deal. There is real value that comes with offering an inspector a fit-for-purpose system that is designed specifically for how they are going to actually review your study. From a cost perspective, from a return on investment, spending the dollars and the time to get that part of it right after you've spent millions on a study is super critical so you don't stub your toes, create findings that frankly shouldn't need to be there because you didn't control the narrative, you didn't control the access...It is really important.

Again, regardless of what system you use to accomplish that, that's something we believe strongly in at Kivo. After seeing customers struggle trying to do this in other, less expensive, perceived cheaper ways,  it's really critical to drive all of this in the right way. 

I'll turn it back to you, Marian, and we'll close this up.

Marion Mays: Thank you so much. Know how your study documents are managed and where they are. Make sure that you followed procedure. If you didn't, write up that deviation so that you understand what you did and how you did it. If something didn't go well, make sure you have evidence of how you managed it.

Ensure the system is fit-for-purpose. Yeah, we say that a lot. Inspectors say that a lot too. They want a fit-for-purpose system. It makes it easier, smoother, less complicated, and it puts more focus on the science and less on the actual process of looking at documentation. And don't forget to prepare your team. They are there to help make sure that this all goes well.

The last word is your trial master file. Remember that it is the evidence that you have about your study. Without it, you don't have the evidence that you need in order to get through an inspection and go to market. It's a valuable asset. Take care of it, and if you have any questions, we're here for you. 

Toban Zolman: Thanks a lot, Marion. And thanks a lot, everyone, for attending. If you do have any questions, please feel free to reach out to us. We are happy to be a resource and help you be prepared for an inspection and make sure that it's a success.

So thanks, everyone. Have a great day. Thank you. Bye.