HOW TO SEE IF AI CAN BE TRUSTED IN R&D DECISIONS | BIOTECHX - JOHN WILEY & SONS INC

Behind every breakthrough, every launch, every approval, is a trial, a test, a surprising connection. You don't just need answers, you need a platform that leads to decisions you can defend. Every successful journey tracks back to reliable evidence, to verified data, to research that holds up. So in science, trust does not happen by believing in a confident voice, but in the ability to be able to check. Any AI that is worth an R&D decision is an AI that you can check. Let me talk about that as a story. So for almost 13 centuries, doctors believed that Galen's anatomical writings as absolute truth. They were unaware that because the human dissections were forbidden, Galen actually dissected a monkey and applied those observations on people. It wasn't until 1543 that a young researcher, Andreas Vasilis, really changed that course and challenged that long-standing authority right here in Basel. And what he was able to do is, by presenting the actual human dissections, he was able to provide the proof that being able to trust something that has not been verified is not necessarily the truth. That skeleton still holds in this city's anatomy museum today as a living proof that direct observation beats blind trust every single time. I am Armaghan Rafat and I run AI and data analytics for a 219-year-old scientific publisher where we've been providing this verifiable proof for two centuries. By the end of this talk, hopefully you'll be able to get one simple test to run on any AI that wants part of your research decision to be able know whether it's working or not, including ours. As the data volumes explode, we don't want for it to be conflicted with a confident voice versus a verifiable prove. So let's get into it. Speaking of data volumes explosion, every 10 seconds, there's a new peer-reviewed research paper that gets published. By the time I walked onto this stage, there were three that were already published and dropped. So that's what we're dealing with. And that's the sort of thing that's reality. To make matters more complicated, The reference lists keep getting longer. On an average count, any paper has tripled the reference size in the past decade. That's not a literature pile. It's a graph. And it keeps getting denser every 10 seconds. So what we did is we did a little bit of research. We asked about 2,400 researchers to see and tell us how they're using the AI. This year, about 84% of them have been using the AI in their daily lives, up from 57% last year. But at the same time, 87% of us, of them, are worried about the use of AI in their day routine. The hallucination concerns are up by 13 points. So, adoption up, caution up, both at the time. That's not a conflict. That's scientists doing the scientific thing. To top the matter off, it's not the problem that people feel that it is and try to solve it with search. Search has been solved for a very long time. Think about this, if you are doing a research in a therapeutic area and you need to learn about a new therapeutic area, it takes months to understand the mechanisms, the pathways and everything else that comes with it. If there is a disease in your neighboring disease modality, how does it transfer? Those are the sort of things that researchers need to think about and they are concerned about these days. It's not a search problem, it's a synthesis problem and it keeps getting worse every 10 seconds. Next slide please. So that reminded me of a quote from Professor Wilson, we're drowning in information but we're starving for wisdom and the world henceforth would be run by synthesizers. People who are able to put the right information at the right time think critically and choose wisely. 28 years later that reads more like a job description for all of us. Two of the words in this think critically, and choose wisely are still yours. Hold that that's not AI that's us making a decision. A bad synthesis in pharma has a price depending upon who's counting a successful drug takes about anywhere from 1.7 to 2.6 billion dollars to be built and still 9 out of 10 drugs that enter the human trials does not get to approval. This is the sort of problem that we are trying to solve with what we are offering today and the issue related to that is the industry's answer to all of that is let's get let's make another click please let's the bigger models the models that hold more context the models rely on the information that is available the problem with that is these models can hold the context they're not making as obvious problems or mistakes that they used to make early on. But at the same time, they don't have the access. And when they don have the axis, they try to reach out to the most available content, which is on web, abstracts, open text, social media. And that's where they draw their information from, which is hard to trust. Another quater problem is that the paper in itself could be accurate. Synthesis could be accurate. The claims could be verified. But it is not until you put 40, 400, or 4,000 papers together that you start to see the linkage between the two that no single paper wrote. Accuracy is on a sentence. Insight is on a corpus. So that created a risk for the teams. A lot of teams start to think about and they say is the AI correct? But some teams otherwise will think that AI is correct, but it cannot give me the answer fast enough. And the problem starts to emerge, is AI grounded in the right set of facts and the right set of information that it can provide? All of you, all of us, have to defend every answer that we are producing to a regulatory body, to our own board, or to your own head of safety. And when you can't defend an answer that is provided by a frontier model or an LLM, it's not an answer. It's a liability with good grammar. And that's what the issue is, right? So what do we do about it? There are three things, and you need all three. Every AI that you use today, run a test and see if that's providing you these sort of abilities. Let's get into the details of this. Let's start with the depth first. When a researcher is trying to find an answer and an AI is trying to reach out, they reach out to the most common sources, abstracts, open text. That's what most of the tools out there are built today. The problem is that abstract is a LinkedIn profile of a paper, 250 words that does not include the kind of dosage that disagreed or the modality that didn't transfer or any other. A peer-reviewed paper that has full text needs to be read by machine in totality in contrast with others to be able to provide the answer that can be completed. Otherwise you're you're operating on with less of an information that you need to. Then the most important part is the human judgment. A peer review includes a human judgment at the start of the paper when the when the reviewer is providing input into the author and then again at the end of it when when somebody is reviewing the actual answer and trying to get the get the details related to it. This is where the decisions become more valuable, this is where the value starts to come in. Remember, think critically, choose wisely. Those are still yours. This is where they show up for any of those. And the last and a very important piece around it is the provenance. Everybody cites a paper. We do too. And you can get citations all over the place. But does the AI have the capability that with one click it can land to the exact passage in the paper to show you what it is that you're looking for and to correlate and have the confidence that the interpretation that AI did is the right one and to be able to utilize those pieces. Those are the foundations that we use to build our applied research intelligence, a platform that we are demoing today and launching as a beta capability right here in Basel. So I'd rather have you look at it and test it yourself then believe me so I'm gonna be brief around these it's built on the foundation of those principles it's built with full-text corpus that Wiley has and with our partners including other public informations but it also built on a patent pending capability that we have built called playbooks not only we can provide you the information but we can help you run workflows and customize those workflows without prompting, without creating any major issues. Ask a question in plain English, see the evidence build, see the review, provide the judgment as part of it and leave the repetition and the work to Ari. That's what we have been delivering. Next slide please. Before I go into a demo, the test that I I promise. Any AI, including ours, that you want to run a test on, ask a question of it that only the full-text corpus can answer. Try to identify a few citations and click on it and see where they land. I have an interesting fact that was really surprising to me. In PubMed three years ago, every one paper in 3,000 will have a link that would not be a real reference. Today, it's in every 277 papers. So a lot of these things continue to grow. And last but not least, try to ask the same question twice or three times and see if the answer is the same. The deterministic nature of the AI starts to create those problems. And if the answer is not the same, can it explain why it is not and what has come in? So let's get into the ARRI demo and see what you think. This is Wiley Applied Research Intelligence, helping R&D teams move from evidence to trusted decisions fast. Provenance makes speed worth having. Here's how ARRI brings them together. Landscape View gives a first read of a whole field, mapping concepts, claims, people, and funding. Before anything runs, ARRI confirms the scope with you, so you stay in control from step one. Here, the researcher asks for a connected view of Alzheimer's disease. ARRI Confirms the topic and sources, then gets to work. ARRI retrieves the papers, awards, and people behind the topic, extracts the claims inside them, and builds a map of how they connect. The landscape is ready. Ari read 411 papers, extracted 1,324 claims, and connected 235 concepts. The map carries the evidence with it. Selecting a concept narrows to the papers, claims, and institutions behind that specific connection. Below the map, Ari organizes its findings into claims, papers, and the experts behind them, each backed by sources you can check. Further down are the grants, institutions, and momentum behind the field, showing where funding concentrates and activity is picking up. Opening the full map turns the field into a timeline. Scroll from 2010 to 2026 and watch the evidence build. Every relationship on the map opens into a ranked list of claims, showing how many papers and institutions support it. Selecting a claim opens its full record. This one, linking amyloid beta to Alzheimer's disease, traces back to 12 papers and breaks the evidence down into causal, correlational, and quantified findings. This is what it means to show your work. The researcher saves this evidence to the session, and it stays with the inquiry. The same paper trail holds across the whole claims table. This landscape now lives inside a shared workspace alongside your team's inquiries and signals. Ari even refreshes it weekly so you can delegate the repetition and keep the judgment. This means faster time to insight, a higher probability of success, and results you can defend because every answer traces back to its source. So, are you ready to get productive? So, I agree with every skeptic in the room who can say, look, the frontier models are cheap. We can rent them and we can utilize them today. That's exactly where opacity gets worse. Just because these models are fluent doesn't mean they have access to the right content and doesn't means that they can provide the evidence that is associated to it. Our CEO, Matt Kisner, says trust is a product. I'd put it more precisely. the product is that you don't have to trust it you can go and check that's where I would say in closing come to the boot run those three tests on us no listen verba you don'T have to take our word for it don't take my word or gallons or any publishers or any AI model come and check thank you