Bio Image Analysis.ORG

Description :

With the advent of deep learning, artificial intelligence (AI) has steadily grown in its use and application in the medical field. Today, AI is being used in several areas of medicine, including disease diagnosis, electronic health records, medical image analysis, and even epidemic outbreak tracking/detection. More than ever, clinicians would like to be able to develop and train their own model for use in their research studies. However, with the plethora of choices available – from the much beloved U-Net, to its many derivations, including the popular Attention U-Net – the choices have become endless. While there are guidelines on how to present AI-driven clinical reports (i.e., CONSORT-AI [1]), no such guidelines or assistance is provided in the choice of AI models or the factors that contribute to the successful deployment of said models.

The objective of this project is to provide an online platform in which clinicians can simply type out the requirements of their research and the type of AI tool they need. A report will then be generated with the stated requirements, allowing the clinicians to then focus on the development of their own purpose-specific AI model.

We currently have a trained large language model (LLM) for this task. The proposed project is an extension of previous works. The selected student will:
1. Test the trained LLM with real clinicians, getting their feedback, and using this feedback to refine and update the LLM
2. Begin incorporating vLLM
3. Extend the current LLM to a Vision Language Model (VLM)

Pré-requis : This project will require a student with a combinatorial skillset, with aptitude and interest in data science, computer science, and systems engineering required for successful completion. Students must have Advanced Skills in the programming language Python.

Travail demandé : Expected Outcome:
The following steps are expected to be completed as part of the project:

1. Background review
o Summary of AI models, their tasks, and common area of applications in medicine
o A review of LLMs and VLMs, and their current applications

2. Data collection:
o Follow previous protocols established for collecting data to augment the current LLM, if required based on clinician feedback
o Use open-source datasets to collect image data
3. Extend work to VLM:
o Train a small, preliminary model based on the initial data available. This model will continue to be updated over time as more data becomes available.
4. GitHub Release:
o The final work will be shared on GitHub, and any accompanying publications completed will be listed in the GitHub release.

Results from this project are expected to be published as a journal or conference paper.


Learning Outcomes:
The students will gain experience in the following areas:
• You will develop critical thinking skills, particularly in the evaluation of the literature, problem-solving in a research setting, and how to present findings to a wide audience.
• You will learn how to communicate research findings clearly and effectively through written reports, presentations, and potential publications.
• You will improve your collaborative skills through continued discussion with mentors, peers, and experts in the field.
• You will contribute to the development of new and novel application, thus enhancing the breadth of the field.