For many small and mid-size Clinical teams, the TMF struggle is real. Maybe you’re using your CRO’s eTMF for now - but what happens after study completion? Or maybe you’re trying to save money by using file-sharing tools - but creating extra work and compliance risk along the way?
It’s 2024. There are better options. And compliant systems to help.
In the following webinar, hosted by Fierce Pharma, we bring together Sponsor, Services and Software perspectives on how to plan and manage the evolution of your eTMF - from active studies to completion and migration from a CRO. Designed for Clinical Operations and IT professionals at small to medium-sized sponsors, this webinar features:
Watch the full recording, or read the complete transcript below.
Rebecca Kushmire, Fierce Pharma: Hello, everyone, and thank you for attending today's webinar, Compliant Trial Master File Migration and Management in 2024, presented by Kivo. I'm Rebecca Kushmire, and I'll be hosting this webinar. This webinar is being recorded and it will be available to watch on demand within 24 hours.
We'd love to hear from you during the presentation. Please submit any questions you have using the questions and answers tab. You can find that on the left side of your screen. Time permitting, we will finish with a Q&A session. Now I'd like to introduce Kevin Tate, Chief Revenue Officer at Kivo, who will be moderating this webinar. Kevin, please go ahead.
Kevin Tate, Kivo: Thank you, Rebecca, and welcome to today's webinar, everyone. We're glad you're here. It should be a fun event. I'm your moderator, Kevin Tate. I head up solutions at Kivo. Kivo has software to help emerging pharma teams manage their documents and processes, including TMF. The TMF in particular is an area where we see small and scaling teams face some real challenges, but also see some real opportunity to explore new approaches. So that's what we'll be exploring today, and we've got a great panel to help us do that.
First we're going to be hearing from Mike Lynch, Senior Specialist at Halloran Consulting. Then we're going to hear from Matt Webb, Executive Director of IT at Elevar. Then we're going to hear from Toban Zolman, CEO of Kivo. I'll be sort of stirring the pot and asking questions along the way.
We've really designed this webinar as more of a panel than a PowerPoint parade, so I do encourage you to use the Q&A tab and send in questions along the way. I think we'll have enough time for this team to share their perspectives and also get into a bit of discussion together. So with that, let's dive in.
I'm going to spend just a few minutes setting the scene and teeing up the discussion. The TMF is something I'm sure we're all familiar with here, and it's not particularly new, of course. The TMF spec has been around since 1996. There's been some changes along the way, most notably in 2018. The EMA guidance to keep this available for inspection for 25 years is something that has affected how some companies plan. But really, a lot of the challenges or new thinking that we see from companies today are spurred on by some other things.
One of those is that the great majority of studies that are taking place today are actually from emerging pharma teams or from small and scaling pharma teams. It's not Big Pharma. So figuring out how do you take this process and the compliant necessities that come with it and address that without big teams and big budgets is a lot of what companies are working through today.
Second is the rise in popularity of working with CROs and other types of outsourcing teams, which is great and gives smaller teams huge leverage. But it also creates some really important management and handoff points around the TMF in particular. So that's something we're really going to zoom into today, is how do you work either on the sponsor side or the CRO side to make sure you're managing the TMF in an efficient and compliant way?
Then perhaps the biggest, most overarching theme is that today's teams in these smaller companies, they're very distributed and they're very dynamic. We're all on a web zoom here, and that's pretty much how we all work. Figuring out how do you do this across time zones when you're not all in the same room or even the same time state together, and recognizing that the teams change over time.
A lot of the companies that we're working with are often in the process of divesting or acquiring assets or partnering with other companies on assets. All that has implications for the TMF. So we're going to explore how some of those forces are at work and how teams manage that today.
Ultimately, the TMF becomes a rather monumental task for those teams. The TMF exists in a lot of different companies and over a very long period of time. So it's not surprising that such an important set of documents and processes become something that a team needs to plan, manage and monitor in a cost-effective way over, as we talked about, ultimately 25 years. So we're going to talk about how teams are tackling this today in terms of where Kivo fits in. I'll spare you the full Kivo commercial - you can check out more about us at www.kivo.io, but this is part of the solution that we provide for companies. In particular, we're going to zoom in on three areas that the Kivo solution helps with today. We're going to talk a little bit about how you can manage TMFs for active trials, both on the CRO and sponsor side. And we're going to zoom in particularly at TMF migration. So when you're done with a study, how do you get that back to the sponsor side from your CRO or from a previous system? Then we'll take a look at longer term TMF storage, again, keeping that 25 year timeline in mind.
So with that, let's dive into our panelists. Mike, you're up first.
Mike Lynch, Halloran Consulting: Great, thank you. Kevin, let's go ahead and jump over to my slide here.
So, as Kevin introduced, my name is Mike Lynch. I'm a Senior Consultant with Halloran Consulting Group. For those of you not already familiar with Halloran, we're a premier Life Sciences consulting firm specializing in strategic, regulatory, quality, clinical and organizational support to both established industry leaders and innovative startup visionaries in the pharma, biotech and med device sectors. You can see here we provide six core areas of expertise, all supporting clinical development: lifecycle targeting, early stage, late stage and post marketing clients. Our expertise areas and core services are those which we are known for have capacity, knowledge and bench strength to deliver. So happy to be here today.
All right. First topic I wanted to talk about was understanding the TMF importance and migration challenges. So the trial master file (TMF) - it serves as the trial's narrative. It's documenting every pivotal moment decision in detail. Its significance lies in the ability to provide a comprehensive and transparent account of the trial's conduct and outcomes. Ideally, inspectors should be able to navigate through the TMF to find all necessary information without the need for additional inquiries. This accessibility ensures efficiency during an inspection and bolsters confidence in the integrity and compliance of the trial documentation.
However, the TMF migrations present a myriad of challenges, each requiring careful consideration and detailed planning to overcome, first being ensuring the chain of custody and Part-11 compliance during TMF transfers. This is critical for maintaining the integrity and compliance of the documentation. It's a process that involves tracking the movement and handling of the trial documents from one entity to another, such as CROs to the sponsor order. Regulatory authorities in the proper chain of custody protocols must be established and followed to safeguard against unauthorized access, tampering or loss of idle trial data, including implementing secure transfer methods and maintaining detailed records of document handling and verifying the authenticity of transferred records.
Moving to the list here. Addressing issues with data transfer methods used by most CROs such as FTP, SFTP, memory sticks and other media...this is crucial to uphold compliance standards and ensure data integrity. In this process, you need to ensure the compliance through validation and other QC activities, providing the option to add documents post trial closure like financial disclosure forms for ongoing compliance. So there is a period of time after the close of the study that you will need to update your TMF. The CRO will hold it for a cost, so generally they're not going to hold it open unless you're willing to have the budget to support that.
A lot of times what we see in challenges is, this is an IT job, in managing a TMF migration. That is not the case. IT is a support mechanism providing the tools and technology oversight for migration. But it's really ClinOps that's on the forefront of getting the transfer and ensuring that the migration is correct all the way through.
Another challenge we see is meeting auditor and inspector requirements for mapping, accessing, viewing and adding documents upon inspector requests, along with preserving the audit trails and metadata.
Kevin Tate, Kivo: I wonder, Mike, as you spoke to a couple of pieces there for Clinical Operations and it needing to work together...Any typical areas of challenge that you see as those groups try to plan and work together on that?
Mike Lynch, Halloran: Yeah, so I'm going to cover this a little bit more in the next slide here, Kevin, but great question.
A lot of times when you're closing out a study and you're getting a handoff from the CRO, there's not a lot of planning that's been done up front. It's okay. Hey, we need to hold this somewhere. It's ours now. Where should we put it? And getting the right tools in place in advance is time consuming.
They need to be validated, compliant solutions. Grabbing it from a memory stick and putting it on Sharepoint isn't a long-term solution. So that's one of the big ones we often see and have to support our clients through. And not only having a solution to do it right, but having a detailed transfer migration plan. You're talking about clinical data. You have to make sure data integrity is enforced all the way through when moving it. So you need a plan to follow that accounts for what I'm going to be moving and how I'm going to test it. Follow it up with a summary after completion of the testing and validation to say I followed my plan. I ensure the documents came over as expected. I can view all of them. Data integrity was intact the whole way through.
Kevin Tate, Kivo: Got it. Thank you.
Mike Lynch, Halloran: Sure.
All right. So moving from what is the TMF and some of the challenges they present, what are the patterns that we often see in TMF migrations and what are the strategies we want to employ to ensure successful migration? One of the patterns that we're seeing is really understanding the importance of upfront planning and purpose definition. As I just mentioned, moving to Sharepoint is not enough. You need to have a clear plan that you can follow, that the stakeholders involved understand, and that resources are allocated to perform those plans is documented to ensure all your TMFs is complete, accurate, compliant and secure during the transfer.
We also see patterns of the metadata and audit trails often being overlooked or an afterthought during the migration. Those are key components of the TMF. Those have to be intact. The FDA's draft guidance on electronic systems highlights the importance of being able to recreate your trial from your digital documentation. You have to have the audit trail to do that, and then additional patterns around validating and testing the migration. As I touched on, you need to go beyond just ensuring that the documents moved over and getting an account of documents. You have to ensure that the quality of the documents was maintained throughout the migration and that the documents continue to be accessible as needed in their existing format.
So then jumping into the strategies here, right, is around understanding the requirements for the sponsor to approve transfer or migration. The CRO is going to hand it off, but the clinical data belongs to the sponsor. They have the oversight, they have the responsibility to ensure it's complete and accurate. So they're the ones who are going to be responsible here. Generally the CROs are going to provide forms for the sponsor just to simply sign off, but those forms are not enough. Generally they need additional documentation demonstrating integrity, data integrity to the inspectors. The CRO is not going to give you that as the sponsor.
The sponsor needs to have that documentation, those controls in place to demonstrate that to an inspector. And as I touched on and highlight again the importance of having a well-defined migration plan in advance of a transfer and your migration summary post-validation activities to ensure that plan was followed and then the migration was 100% accurate and successful. This needs to be authorized, approved by ClinOps, IT, whoever those key stakeholders are that are designated in performing the migration and validation responsibilities, that's who would approve that document.
Kevin Tate, Kivo: Got it. Big theme around really thinking through everything right up front and being able to look all the way to the end before you start the process, I'm curious...you highlighted a couple of times that just using Sharepoint, frankly, isn't really enough, doesn't serve the needs. Maybe help us understand more what are the risks there? I think a lot of teams are probably using some of these tools and wondering how to quantify those risks in able to do that planning or, I don't know, allocate the budget to it. Help us understand those risks.
Mike Lynch, Halloran: Certainly. So generally, SharePoint is not a validated system. You're talking about clinical data that has to be maintained in a validated environment. Really the tricky part around SharePoint is around access controls and permissions. Those are a little more cumbersome to enforce. It's not going to come out of the box like a lot of systems that say they're GXP compliant. There's additional rigor that you're going to have to employ as an organization if you want to consider that compliant Additional user access controls, Kevin, is key.
Folder restrictions are a little bit lax in Sharepoint. So now you have documents and data that know for the clinical trial that people that may not necessarily supposed to have access to that can get to it in Sharepoint.
Kevin Tate, Kivo: Got it. Well, I think obviously one of the advantages that you and your team have is you get to work with a wide range of clients on this and probably at a lot of different points in the process. As we talked about the beginning, there are some particular challenges that smaller teams face. I wonder, do you see any pattern recognition around how smaller teams are looking at this today versus maybe how they have in the past, or what are some of the trends you see, I think, in this segment of the market?
Mike Lynch, Halloran Consulting: Good question, Kevin. So some of the larger, more mature organizations, they'll have the expertise and more robust procedures in place to support such data migrations and transfers. But one of the things we see with our smaller kind of newer clients is they don't necessarily have the resources and especially the documentation in place to enforce those procedures all the way through. So we're seeing a trend in really focusing on building out what those procedures are, especially as they get closer to receiving the TMF from the CRO.
We look at a lot of things from a risk-based approach. So looking at our clients, where are they in their clinical journey? Is this a phase one or phase two trial? Then the risk of inspection isn't necessarily as high as a pivotal phase three trial where you can guarantee that the documents and data are going to be inspected.
Kevin Tate, Kivo: That's great. Well, thank you very much. We're going to now shift on and hear from Matt at Elevar, and we'll get back to the group discussion at the end. Thank you very much, Mike.
Mike Lynch, Halloran Consulting: Thank you.
Kevin Tate, Kivo: All right, Matt, it's your show.
Matt Webb, Elevar Therapeutics: All right, thanks, Kevin and Mike, excellent points that I have a little bit of commentary on as well.
First, a little background on Elevar Therapeutics. We are one of these small emerging biotechs that we're talking about here, that face these challenges with having smaller groups, having only a couple of people in each function that are specialists, and not having all that bench, as Mike alludes to, that they have to help with these kind of projects. We have a PDUFA date right around the corner. We're looking at commercializing our products. We have two products that are combo therapy that we're going to be looking at treating HCC, ideally with a great deal of success.
Along with that comes some challenges, and especially around this timing with PDUFA right around the corner. Just as Mike was mentioning, we're going to be facing inspection. So inspection readiness...is all of our TMF data in order? Do we know where it all is? If an inspector says, "hey, show me this file from this trial," can we pull it out and are we able to back it up and say it's got that integrity behind it, which I'll touch on a little bit more.
So, some of our challenges...we have a lot of trials that have gone through historically different languages, different CROs, paper, electronic...trying to get all that data together, trying to harmonize that data into the kind of packet of information that you're going to need to submit to the agency, that you need to have ready when they come in for inspection, is really challenging. Another challenge is, the strategy that clinical might have between insourcing their clinical efforts versus outsourcing their clinical efforts and leveraging those CROs, that can involve migrating data out of the organization, migrating data back into the organization. All of those data integrity points are super critical with every single movement of a piece of data.
Managing the costs is another great point, which these small biotechs like the ones that I've pretty consistently worked for can't afford a large TMF solution and can't afford to have a team of TMF specialists in-house. We all know how biotech works. Funding comes, funding goes. This is just the roller coaster that we live in. So managing that is really important as well.
Active and archival cases are important as well. We have ongoing trials. We need to have clinicians and clinical groups actively entering data into that TMF. But we can't ignore the fact that we have all this historical data that also needs to be kept with the same degree of rigor that that active TMF data does. So a number of unique challenges that every organization faces, but smaller organizations like ours have even more challenging time with.
So the solution...why did we go with Kivo? Why did we choose Kivo? What stood out?
We are a SaaS-first organization. Again, small groups, small IT functions, small other functions. SaaS is really important because then my team doesn't have to build up a bunch of servers, load operating systems, worry about all this stuff, and then on top of that, install applications, et cetera. That's just a non starter for small groups. Kivo understand the challenges of the SMB space, of the emerging biotech space, and they showed that in spades with our first engagement with them, migrating a number of TMF documents from disparate systems into theirs. The ease of use versus complexity is it's refreshing.. There's not even a word that expresses how refreshing it is, because the complexity of Part-11 systems going back decades, it just doesn't make sense. They were designed by and for large organizations. There's too much to do, there's too much data to enter, there's too much backup and all these other things to consider that, again, why, if we can do it easy, let's do it easy. And that's what I really love about the Kivo platform.
Additional boxes that they check...Part-11 Compliance, which is table stakes. That's got to be there. Having a responsive design, knowing that I can go on my PC or on another PC, on a Mac, it doesn't matter. It all looks the same on this nice web interface, making it easy for the users to get into. It's economical, it meets our price points.
And again, inspection and CRO access. If an inspector comes to us and asks for that data, I don't have to go to a fireproof safe, pull out a USB drive, load it somewhere, try to print a file. I can just grant an inspector access to our platform with those controls and permissions in place that Mike was alluding to that are really important for that separation of access.
It was funny that you mentioned SharePoint..you CAN make SharePoint Part-11 Complaint if you really want to make it completely unusable as a collaboration tool. I've seen it happen! I've found TMF data files in Sharepoint before. I won't mention the organizations or the groups responsible, but they end up there. And so then what do we do now, and how do we get those things out of there? How do we get something from a CD-Rom that we got from a CRO? I know that's an antiquated term, but a CD-Rom or memory stick, all these files, that was the way that we had to do it because we couldn't afford a big box TMF solution where a CRO could plug that data into.
Another good point that was raised - we can't afford to have the CRO continue to manage this and retain it for now up to two and a half decades. It's unfeasible. Your retention requirement would go well beyond the relationship term of a CRO. So that provides some unique challenges as well.
I did want to touch on the data integrity because that is a really important one. We just recently experienced this, where the CRO, unnamed CRO, provided us with an export of our TMF and the numbers didn't line up. We ran the checksum values. It just didn't look like the data integrity wasn't there. And after I don't know how many meetings, we finally identified that the CRO was giving us a different version of the TMF data than we really needed to have that audit trail to have all of that historical information and metadata behind it. So we finally were able to hammer that out thanks to that being identified. If that weren't identified, then we'd have this whole bolus of data sitting in our TMF that might not be valid. So those are just some of the additional challenges that we face.
Kevin Tate, Kivo: Well, thank you, Matt. We certainly enjoyed working with your team on the TMF and on these migrations.
So a couple of questions. We met in the first place because you were looking for a different type of solution, a different approach, and I'm curious, what sparked that? Was there a reason that you decided or realized that you needed a different approach and went out looking?
Matt Webb, Elevar Therapeutics: I didn't want to get CD-Roms or memory drives in SharePoint.
Kevin Tate, Kivo: No more CD-Roms!
Matt Webb, Elevar Therapeutics: The biggest thing was the long term storage; that was the impetus for finding a solution. There were a number of old CROs where we had some data stored, and it was Part-11 compliant storage, but not a TMF platform. So it didn't have any of the TMF structure that you would need to effectively both put and retrieve data into a solution like that. And it was just a happy coincidence that Kivo also supports active eTMF.
So not only did we find a solution for our archival data, but should we engage in future trials, post market trials, et cetera, we now have a great solution where we can start the data there, grant the CRO access, and they can put stuff there, or we work with the CRO. We have the understanding that at the end of that trial, that data is going to get bundled up and go directly into Kivo. That was the impetus for it.
Kevin Tate, Kivo: Got it. A couple of great questions coming in from our live studio audience. So let me get to those, because they're about what you were just covering.
You talked about being able to use the checksum to identify that the data wasn't accurate and wasn't going to match your records. Can you talk a little bit more about that? Got questions about how the checksum played in and how you were able to identify the mismatch.
Matt Webb, Elevar Therapeutics: I will speak to it as well as I can. I will admit that I'm not a data scientist by trade. I'm an IT leader. So my days of knowing what checksums are are a little behind me. However, a file that it's created along with a checksum value, that's like its signature, this is that file's fingerprint. If that file is modified, if it's copied into a new system such that it gets a new modified date, if a new person works on it, even if you rename the file, all of those things will change the structure of the file and it will get a different checksum value. So now you know, "Oh, okay. This file has been modified even if it doesn't look like it", even if none of that's visible to the naked eye, you know, this is not my original file. This may have data integrity issues.
Kevin Tate, Kivo: Got it. Thank you. That's helpful. A couple of questions came in related to the reality of using SharePoint and then moving to a more Part-11 Compliant system. For example, is there a path where you can use SharePoint and then upload to or migrate to a Part-11 compliant system, not for your TMF, but for your other documents as well? I suspect this is something Mike and Matt that you've both lived different versions of. So maybe talk a little bit about what life after SharePoint can look like.
Matt Webb, Elevar Therapeutics: That's a tough one. Mike, I don't know if you want to bite. My quick comment is you can, but you have to have a really compelling story behind that or attached to that that says, "here's where all of our data came from, here's what we did with it, and here's where it lives now in its nice, happy place to be" Mike, can you expand on that?
Mike Lynch, Halloran Consulting: I would avoid that situation because now you're talking about a migration to two different places, right? You're moving it from source system to SharePoint temporarily to house it. So now you put it in a non-validated system. Now you're going to have to do a migration again to move it back out. So you've kind of double stacked against yourself there in terms of housing it at a compliant location.
Matt Webb, Elevar Therapeutics: One final point with non-TMF. So, for example, in EDMS platforms that are part of your Quality Management System, we can work on documents outside of a Part-11 system. You can collaborate on documents because sometimes, depending on the system, EDMS solutions are not very collaborative by design. So if you need ten people commenting on a document, that might just be better done in SharePoint, and then you can upload it, go through the appropriate workflows and the appropriate process within the Part-11 system. So it's not to say that it's always, "You can't do this outside of Part-11", but you need to have defined processes, defined workflows. That's where you bridge the gap, is with how you describe what you're doing.
Kevin Tate, Kivo: And I can't help but say, to your point, the ability to use Kivo to edit and collaborate in Microsoft Office but manage the documents in Kivo is one of the things that people often are looking for when they're trying to move past SharePoint.
A couple more questions specific to migration that I'll cover, a nd then, Toban, we're going to get to your section. One is what about TMF migration for a study that was terminated early, especially if there's intention to use the safety data for another study. Any experience with that or advice around that?
Mike Lynch, Halloran Consulting: Yeah, I can take that one there, Kevin. So if there's the intention to use that study data in a future study, we would recommend go ahead and archiving the TMF for that particular study so that data stays compliant, it stays accurate, complete, follows data integrity policies, but that can be later retrieved for future use.
Toban Zolman, Kivo: The other thing I would tack on to that is, we see that quite a bit with small companies where they may have ended a study early, but they plan to divest whatever IP or assets they gained from that. And so having that in a system that you can easily transfer out, you have full chain of custody for all of that data for the entire duration of it...That becomes critical to try and either recover data for future studies or divest it to another sponsor and try and recoup costs.
Kevin Tate, Kivo: Yeah, great point. Okay, we're going to keep cooking. Toban, you're up next. I know there's some more questions than the queue. We'll make sure we get to those. But up next, Toban Zolman.
Toban Zolman, Kivo: Yeah, thanks, Kevin. I appreciate it.
So I want to talk just a little bit at the beginning here about kind of a typical TMF timeline. We've addressed this. I think all the speakers have at some level on the challenges of supporting that kind of 25 year retention period. But at Kivo, we've really zeroed in on what we call "the TMF gap". This isn't a term we created. If you've been to a TMF conference in the past decade, you've probably seen some version of this slide. But this area in red that we've highlighted is really critical because you've been through the live portion of a study the trial has completed, and in many cases, the CRO stops managing that and it transitions back to the sponsor.
In that stage, you may have TMFs in multiple formats. They may be on non-secure media, not easily accessible or able to be inspection ready. The challenge that we've seen, especially post-pandemic, is that inspection period where inspections are most likely (you can see in that kind of gradated stripe on the bottom) are super critical. And most of those inspections have moved virtual. So being able to tee it up so that an inspection can happen and happen in a system that is compliant and accessible outside of premises of the sponsor is super important.
What we've really tried to focus on with the Kivo platform is three areas in managing that transition from the CRO to the sponsor, and I'll talk through this in terms of management features and just how we think about this, but really being able to, a) take possession of it, b) do any additional management in terms of documents that are received late (site close out materials, et cetera) and then transition that into long term storage, and then c) also be able to support virtual inspections. Those are the key areas that we've really narrowed in on, in large part because for emerging companies, those are the largest areas of pain, and they may not have a system in place, nor can they afford a traditional system to manage all three of those aspects in a way that really is viable, especially when you throw in requirements around virtual inspections.
So to talk about the process and the way that we tackle this...the first step that we often look at is what we would call migration. So being able to manage the chain of custody and preserve compliance as a study moves from the CRO into the sponsor. This is an area, as I said, everyone's already talked about this at some level in this webinar. But the real challenge is that the incentives of the CRO are not necessarily aligned to the incentives of the sponsor. At this stage of the project, the CRO is likely paying for a TMF system. They are paying per study, per month, and it is in their best interest to exit that cost as soon as the contract completes or to upcharge the sponsor radically to cover those costs. So what we see happen a lot is that what CROs are prepared to transfer to a sponsor is not quality.
I would say if you look at a typical TMF migration project that we execute at Kivo, two thirds of it is just getting the right data from the CRO. And I would guess 80% plus of those transfers are not correct the first time. So it doesn't matter how well we define what is needed in terms of audit trails and metadata, we run into issues like what Matt was talking about with checksums aren't right, or they're including the incorrect renditions or the wrong versions, or they've omitted approved documents, or, I mean, the list goes on. So really, what we've tried to focus on from a product standpoint and an offering standpoint is managing that chain of custody.
The second thing is, once we have it in our possession -and I would say, the way in which we do that is unique. Most transfers happen from the CRO to the sponsor on media, or maybe it's downloaded. And the second that happens, it's out of a compliance system. So in Kivo, we've basically built this so it goes from the CRO system directly into a secure backend in Kivo, where everything from that point on is managed via access controls and has an audit trail. That greatly simplifies the migration plans that need to be built. And as part of that, then we preserve audit trails and metadata.
And when I say we preserve those, I really mean that. A lot of systems are set up where they'll ingest the audit trail so you end up with a file and then an audit trail file setting next to it. If that TMF moved from multiple systems, you have a document with two or three different audit trail files, plus anything that happens in the new system. We actually parse those audit trails on import so that if you look at a file in Kivo and look at the audit trail, you can see everything that happened in the first CRO system, and then the second CRO system, and then the migration into Kivo, and then anything that happened in Kivo to consolidate all of that into a single view.
I call this automated adaptability. Because really what we're doing through this migration is we're trying to take a view of data standards, where it's something like EDMS or even export from a system like Phlex or Veeva or whatever it is, and be able to take all of the variations that you might get from that. Because no two exports are the same. Trust me, an export from those systems is like a snowflake. You're never going to find two that match perfectly because they just have so many options and configurations before you ever even get to the export. So what we've built is really a sophisticated rules engine on imports that uses what we know about each vendor and then lets you do mapping.
We talk about lossless imports as part of this. We've built a new type of a backend where we can ingest any type of data and store that in Kivo as a key value pair, and have this large ability to store unstructured data that way. What that lets us do then, is if in six months or five years or 20 years, there's data that was in the source system that becomes relevant for some reason as part of an inspection or something like that, we can promote that into metadata and we can query against it anytime. Really retaining literally everything from the source system, not dropping anything on the floor, is super critical.
Data transformations are part of this, as we may move from one TMF version to another. We may take metadata or naming and augment that with richer data that maybe wasn't in the source system. Then when we do all of this stuff, including data integrity checks, as Matt was referencing, we can take that recipe that we've built and reuse it.
That really creates economies of scale for us. We can run a migration in a staging or a sandbox environment, do QC on that. If everything checks out, we can rerun that in production using the exact same recipe. If we are getting multiple TMFs from a single CRO, we can build a recipe specific to them and then just tweak it as the snowflake changes from study to study. All of that becomes really critical to automate the import in a controlled and validated system, but be able to be flexible and adaptable as there are changes, intentional or otherwise, to the data that we receive beyond migration.
Yeah, go for it. Kevin.
Kevin Tate, Kivo: Quick question came in about migration. Before you move on, the question is, if you have two TMFs. So if a CRO has two TMFs for one study, is it possible to transfer those two different TMFs over to Kivo, even if they came from different systems? And how would we do that?
Toban Zolman, Kivo: The short answer is yes, we can take stuff from multiple systems. We can merge TMFs together. That is a scenario that we see frequently, frankly, where the sponsor may have sponsor maintained documents, the CRO may have site maintained documents, and we combine those to create the full TMF. So yeah, there's a multitude of ways we can do that, depending on the scenario.
Kevin Tate, Kivo: Got it. And then in terms of verification and validation activities, would it be best practice to perform those for each migration location if your data is moving into different systems? Maybe this goes back to Matt and Mike as well...what's the best way to think about verification and validation activities as the TMF goes through different systems over time?
Matt Webb, Elevar Therapeutics: Data integrity checks definitely need to happen. I think that's the most important thing, because you don't know what environments you're coming from. So being able to verify and validate one location, Sharepoint, for example, isn't possible. But you can still do data integrity checks to make sure that what you're getting is what you're talking about frequently.
Toban Zolman, Kivo: We do see this frequently where we'll be told the TMF is sitting in this system, and it is, but that's not where it started its life. And the sponsor doesn't actually know that it actually started in SharePoint and went to Phlex and then went to Veeva, and then we get it and have to stitch all of that together.
So I think what we have focused on from a system standpoint and a process standpoint is being able to handle the transformation and the data integrity checks from any source system and so that means we can kind of build a bit of a tur-duck-in, if you will, a TMF tur-duck-in, where we can validate what came from System A and then how that affected System B, and then take that final migration and put into Kivo and consolidate all of that into one. So turducken is a technical term that gets used a lot.
Kevin Tate, Kivo: Now it sounds like, Toban, our T-shirts are going to need to say "we checksum turduckin" on the back, and I think we're good. So problem solved. Okay, thank you for that. Take us on to manage, Toban.
Toban Zolman, Kivo: Sure. To talk through the management here real quick, we spend a lot of time and effort doing the migration from CRO to sponsor, but that doesn't mean that the management of the TMF is complete at that point.
We do handle active trials in Kivo. We have a lot of companies who run their trials in Kivo, including giving investigator sites direct access into Kivo. We're seeing more and more of our customers manage the TMF in Kivo and give the CRO access because it is more cost effective, and frankly more intuitive, easier to collaborate, et cetera. So I think that model is starting to change, especially for emerging companies, because it puts them back in control. But we give you the ability to manage that, whether it's an active trial or once you've received it from the CRO, to handle late arriving close-out documents. As I said, we really built Kivo post-pandemic to be for virtual inspector to be a first class citizen, if you will, in the product. So we have specific roles for that that facilitate virtual inspections, customize their view, and give the sponsors the ability to kind of choose what they see and how they see it.
The final thing I would say is, we don't want to be part of the problem. We've talked about all of the challenges getting data into Kivo. We've also made it incredibly easy to get it out of Kivo. We're often used as a pass-through system to even normalize TMFs as they come from half a dozen different CROs and ultimately need to go into one sponsor-owned system or need to get divested; all of those get imported into Kivo. Audit trails, metadata structure, all gets normalized using our tools in a validated way and then passed off to another system, either as part of a merger, acquisition, divestment, you name it. So that management piece is critical as stuff moves through. Which kind of leads to the final step for us, which is retention.
This is an area that I think is challenging for a lot of emerging companies because they don't have permanence with their systems, so to speak. They may have just hired somebody like Matt to run it, and so they don't really have a system that they can put a stamp on and say, "our data is good in this for 25 years". We do data integrity checks for any study that gets closed in Kivo and set up for long-term storage. Those data integrity checks run every quarter, so that if there is an issue discovered, and that issue would be like... We're using cloud storage. If there's an underlying hardware failure in the cloud, this gives us the ability to identify that, go to backups of that data, restore it, and ensure that at any given point in time, you've got a fully compliant TMF and a fully compliant system, and you know that the integrity of all of the data that's sitting there is going to be maintained over time. So that's really how we try and look at the process. Those are the challenges that small companies have, scaling companies have. So we've built features to really support each of those steps in the process.
Kevin Tate, Kivo: Thank you, Toban. A number of additional questions came in. In terms of strategies for transferring from the TMF from CRO to sponsor, versus transferring a TMF from one TMF vendor to another TMF vendor, is that effectively kind of the same thing, the same approach? Or are there differences about CRO to sponsor versus system to system?
Toban Zolman, Kivo: From a technical standpoint, we treat them the same way. I think if you're going from system to system and you're in control of both systems, it gives you more levers to pull to ensure that the data comes over smoothly. We help customers dig out of a lot of challenging situations where they may have had a study that ended 18 months ago, it's been sitting in a secure file share or something like that, and then we go to migrate it and turns out there are data integrity issues. It's missing audit trails, it's missing metadata files. Whatever the case may be, we just have fewer levers to pull to remediate those issues. And that becomes much more of a battle sometimes between the sponsor and the CRO to reconcile that stuff. So it's much easier when you have access to both systems and can ensure that what's coming out of the source system is clean. But from a technical standpoint, there's not a big difference to us. It's really just how you manage the process and the relationship in that.
Mike Lynch, Halloran Consulting: To add on, hhave a well documented plan that you can follow, that you can document, and show that the history of the transfer was recorded, the chain of custody was recorded, and that the plan you set out to follow was performed.
Kevin Tate, Kivo: Very good. A couple of other questions here about Kivo in particular. One is, is our bread and butter really more end-of-study versus active study? No, we do a lot of work with active study as well. We've really tried to zoom in on this gap and the TMF challenges around that, but we absolutely do a lot of work around active studies.
Then we got a question that's always a tricky one to answer, and I'm not going to ignore it, I'm going to answer it, which is, can you give us a sense of cost versus other systems? Rather than just saying it depends, I will say it does depend a bit on team size. But to your question around comparisons, most of the teams that use Kivo have an annual subscription that's more five figures rather than the six figures or seven figures you might see from some of these other systems. Specific to TMFs, most of our TMF transfers end up being projects that take around a few thousand dollars and require a few weeks. Those, we found, are budgets that tend to align better to small and scaling pharma teams.
Toban, you're my boss. You can shoot me later for answering that question.
Toban Zolman, Kivo: No, that's pretty accurate.
Kevin Tate, Kivo: Good. Let's see a couple of questions. Let me take a look at the priorities here. Very good...Covered that... So a question zooming back up would be in terms of advice. We talked a lot about the sort of challenges and approaches specific to different teams and stages here. If we go back to the beginning and a team is trying to plan ahead and think about what's the right long term approach, what advice might you have for a team as they start that planning? How would you encourage them to, in particular, perhaps have clinical operations and IT work together, as has been a theme today around that planning? Any advice?
Matt Webb, Elevar Therapeutics: I'll start with the IT working with clinical. It is really important. I mean, IT typically has their own strategic roadmap. They have their systems in place from data management strategies, single sign on, identity management...there will be a lot already that can impact your selection of TMF vendor. I would say come up with your strategy for working with the CRO. What are you willing to let them manage? What do you want to manage? Those are kind of my critical ones. Mike?
Mike Lynch, Halloran Consulting: Yeah, I'll second that, Matt. So, yeah, frequent conversations with your CRO, upfront planning with them, making sure that expectations are aligned across those, and regular communications on what that's going to look like for the transfer and what those requirements would be on your side to perform that transfer.
Toban Zolman, Kivo: I would add just one quick thing to that, which is we've seen projects go best when there is a tight coupling between a technical resource -typically IT, but it doesn't have to be - but a technical resource and ClinOps, and that that coupling happens prior to the imminent delivery of the TMF back to the sponsor. What we see happen a lot is CROs...What they're delivering back may not be technically correct. They may not even know that. It's not like they're trying to pull a fast one. They're just not savvy enough to know what they're delivering. Without a technical counterpart on the sponsor side to really drive that handoff, it gets really challenging. We end up stepping into that role a lot.
It's challenging for us to navigate a CRO sponsor relationship and ensure that everything is happening correctly. So having a technical person in the mix on the sponsor side and then giving yourself time and for there to be some grace in that process to get stuff figured out is critical.
Matt Webb, Elevar Therapeutics: And think about the end. So often when you're kicking these things off, you're thinking about how do we set up our TMF? How's it going to be structured? Where's our data going to live? We're not often fast forwarding to what's inspection going to be like, how are we going to pull this data out? What if we have to migrate it? If you can get enough leadership involved that you can get those conversations around, "Well, we're starting with this, how are we going to end?" it can make that process and your strategy just a lot more linear and less prone to hiccups and bumps.
Kevin Tate, Kivo: Thank you. Thank you, Matt. A couple more questions came in that we'll squeeze in here before time. One is, where does paper scanning fit into all this? Right. We haven't used the P-word yet today, but paper is a reality of the history here. Any best practices around that or how that's played into these?
Toban Zolman, Kivo: I can maybe take a first stab at that. So we don't see it a ton. For large scale paper trials, the challenge these days is getting it scanned. There are not very many vendors left in the US that do large scale scanning projects, and so there's very much a supply and demand curve on that because it's down to two-ish major players. Pricing reflects that. There's not a ton of competition on that side. So that's an area that we have seen some of our customers struggle with, especially if they've acquired a program that has paper TMF and they want to get it into a system to meet standards. It's really just, how do you solve the labor part.
The technical piece is easier because there's nothing there. There's not audit trails, there's not metadata. It's just really a bulk, import mapping exercise. But for sure, if you have something to scan, I would recommend doing that now versus in three, five years, because it's just getting harder and harder to find folks who do that work.
Matt Webb, Elevar Therapeutics: Yeah, once you've scanned them, the conversation becomes the same. Then it's just data, to your point, minus the metadata and audit trail. But we've had that because, as I mentioned, we have a ton of historical trials going back quite some time. So there is an amount of paper records that are stored off, and we've had to have some of those scanned in by some of these major providers. And it's painful, expensive, time consuming. So now is better than later.
Kevin Tate, Kivo: We got a question about Kivo's QMS module, and speaking to that, Kivo does have a regulatory module and a QMS module as well. We won't go deep into QMS here, but it does handle SOPs and training and things like CAPAs and deviations and other quality activities. So more information about that on our website. But a great call out because, ultimately, having a consistent DMS and having all that be integrated is a big piece of the puzzle here.
Perhaps our near final question is, if there's no technical IT resource at your sponsor team, how do you think about working with a consultant or working directly with a solution like Kivo? And I will say, from our perspective, it depends. But a big reason that we partner with teams like Halloran is that consulting expertise is super valuable, not only from a technical standpoint, but from a process and planning standpoint as well. And so I think having some expert resource in the mix is important. Mike, maybe talk a little bit about how companies choose to work with an external IT consultant like that.
Mike Lynch, Halloran Consulting: I appreciate that, Kevin. We provide this service quite often. Being that we have a data and technology service area, but we also have the clinical service area...It's kind of our secret sauce. The combination of our resources and bench depth combined together can help manage the ClinOps side versus the IT side as well. Like I said, this is actually an area where we do see quite a bit of client work.
Kevin Tate, Kivo: Great, thank you. Last question, and then we'll wrap up. If we do have an online system, we do want to upload a whole bunch of documents...Is there any limit to the number of documents that we can import? And excited to say, there is no limit. There really isn't. We often import TMFs that have over 100,000 documents in a single study, and a lot of times we're pulling in 10+ studies. So, yes, no limit.
All right, thank you so much to our panelists. Thank you to the audience for joining. And with that, I think it's a wrap. Rebecca?
Rebecca Kushmire, Fierce Pharma: All right, well, thank you again. And I'd like to thank everyone for attending this Fierce Pharma webinar and for submitting so many great questions. I'd like to give another thank you to our speakers for attending and participating, and Kivo for presenting today's webinar. A recorded version of this webinar will be available for you to access within 24 hours using the same audience link that was sent to you earlier. Thank you again for joining us, and we look forward to seeing you at future events.