Dataset Analysis & Statistics

Comprehensive analysis and statistical overview of our medical datasets including detailed comparisons and key insights.

Medical Dataset Analysis & Statistics

This comprehensive analysis covers three key medical datasets, providing detailed insights into their structure, content, and distinctive characteristics.

📊 Dataset Overview

Our analysis covers three distinct medical datasets, each serving different purposes in the healthcare AI domain:

General Medical Instruction

1,867 entries focused on educational medical content with concise questions and detailed explanations

Evaluation Medical Instruction

3,324 entries designed for medical exams with multiple-choice format and brief answers

GenMedGPT-5k

3,088 entries simulating patient-doctor dialogues with focus on pain management

---

🔍 Detailed Statistical Comparison

Text Length Analysis

Dataset	Entries	Avg Instruction	Avg Input	Avg Output	Input-to-Output Ratio
General Medical	1,867	4.0 words	17.10 words	33.10 words	0.52
Evaluation Medical	3,324	46.0 words	30.05 words	2.67 words	11.25
GenMedGPT-5k	3,088	15.0 words	23.04 words	44.76 words	0.51

Top Medical Terms by Dataset

General Medical

Input Field Terms:

• syndrome (8.94%)
• disease (6.37%)
• carcinoma (3.00%)
• infection (2.46%)
• tumor (1.98%)

Output Field Terms:

• characterized (high frequency)
• type (high frequency)
• disease (high frequency)
• syndrome (high frequency)
• cells (high frequency)

---

🎯 Key Distinctive Features

1. General Medical Dataset

   Educational Focus

   • Balanced medical terminology distribution
   • Moderate input complexity with detailed explanations
   • Strong emphasis on syndrome and disease classification
   • Perfect for medical education and reference systems
2. Evaluation Medical Dataset

   Examination Format

   • Extremely high input-to-output ratio (11.25x)
   • Long, complex questions with very short answers
   • Frequent use of “except” indicating multiple-choice format
   • Ideal for medical board preparation and testing systems
3. GenMedGPT-5k Dataset

   Clinical Dialogue

   • 34.62% pain-related content - extraordinary specialization
   • Patient-doctor conversation simulation
   • Longest output responses (44.76 words average)
   • Optimal for conversational AI and patient consultation systems

---

📈 Medical Condition Prevalence

Cross-Dataset Condition Analysis

Pain Management Focus

GenMedGPT-5k shows extraordinary specialization:

• pain: 34.62% (vs. 0.37% in General Medical, 3.09% in Evaluation)
• This represents a 90x higher pain focus compared to other datasets
• Indicates specialized use case for pain management applications

General Medical Conditions

Balanced distribution across datasets:

• syndrome: 8.94% (General), 4.87% (Evaluation), 1.42% (GenMedGPT)
• disease: 6.37% (General), 6.17% (Evaluation), 2.81% (GenMedGPT)
• blood: 1.50% (General), 3.22% (Evaluation), 3.33% (GenMedGPT)

Specialized Terms

Dataset-specific terminology patterns:

• General Medical: High carcinoma/tumor focus (medical education)
• Evaluation Medical: Balanced clinical terms (exam preparation)
• GenMedGPT-5k: Patient symptom language (consultation simulation)

---

🔗 Correlation Insights

Input-Output Length Relationships

General Medical

Correlation: 0.1423

Weak positive correlation between question length and answer detail

Evaluation Medical

Correlation: 0.0392

Nearly no correlation - consistent short answers regardless of question length

GenMedGPT-5k

Correlation: 0.2587

Strongest correlation - longer patient questions receive more detailed responses

---

🎯 Dataset Selection Guide

Choose the Right Dataset for Your Use Case

Medical Education

Recommended: General Medical Dataset

• Balanced medical terminology
• Educational question-answer format
• Detailed explanations and definitions
• Covers wide range of medical conditions

---

📋 Quality Metrics Summary

Content Diversity

Excellent

25+ medical specialties covered across datasets

Format Variety

High

Educational, examination, and conversational formats

Specialization

Strong

Each dataset serves distinct use cases effectively

Correlation

Varied

Different input-output relationships match intended use