Project LITe Survey Automation Server

Overview

                The Challenge: Large longitudinal studies require constant tracking, status updates, and outreach.
                
                The Solution: A locally-hosted automation server that ingests REDCap data, keeps a tracker up to date,
                applies visual status rules, and prepares multi-channel outreach — scheduled and repeatable.

Python

REDCap API

Sheets API

Email

SMS

Pandas

Cron

System Architecture

1

REDCap data ingestion

Pull completion/status via API • pivot by participant/event • write clean exports

↓

2

Tracker sync

Update a central tracker automatically so coordinators see the current state

↓

3

Visual status rules

Apply rule-based highlighting so missing/completed work is obvious

↓

4

Outreach

Generate links + messages • deduplicate • deliver via email/SMS

Core Modules

📊

FollowupCheck

Pulls completion data from REDCap, reshapes it into a participant-by-timepoint matrix, and syncs it to a central tracker.

# Fetch REDCap report and pivot data
def fetch_and_process_redcap_data():
  payload = {"content": "report", "format": "csv"}
  response = requests.post(api_url, data=payload)
  data = pd.read_csv(StringIO(response.text))
  combined = data.pivot_table(index="record_id", columns="event", aggfunc="first")

REDCap API

Pandas

Sheets

🎨

Conditional Formatting

Applies rule-based color highlighting to the tracker so coordinators can triage quickly.

# Apply conditional formatting rules
def apply_conditional_formatting(service, sheet_id):
  # add rules using the Sheets API
  pass

Sheets API

Rules

📧

FollowupText

Generates personalized links, composes reminders, and dispatches them via email and SMS with deduplication.

# Fetch personalized survey link
def fetch_survey_link(record_id, event, instrument):
  payload = {"content": "surveyLink", "record": record_id}
  response = requests.post(api_url, data=payload)
  return response.text.strip()

Email

SMS

REDCap Links

🔄

Scheduler

Identifies who is due next and prepares future sends/imports using date logic and enrollment data.

# Schedule next wave (example)
def compute_next_date(base_date):
  return base_date + timedelta(days=31)

Date Logic

REDCap Import

Technical Workflow

Scheduled execution:
                    The system runs on a schedule (cron) that is tuned per deployment. Example below is illustrative.
                

# Example cron schedule (illustrative)
0 2 * * * /usr/bin/python3 followup_check.py
15 2 * * * /usr/bin/python3 apply_formatting.py
30 2 * * * /usr/bin/python3 scheduler.py
0 8 * * * /usr/bin/python3 outreach.py

🧬 Project LITe Survey Automation Server

Overview

System Architecture

Core Modules

Technical Workflow

Demos

Automation server (visual preview)