Frozen Snapshots & Test Archival

Quick Summary: Frozen snapshots create immutable copies of your test files that are locked to your study, ensuring participants always run identical test versions even if PEBL updates its test battery. This guarantees research reproducibility and protects your data integrity.

What You'll Learn

• What frozen snapshots are and how they differ from regular snapshots
• Why frozen bundles are critical for research reproducibility
• When frozen snapshots are created automatically
• How to handle battery updates when tests change
• How the frozen bundle system protects your data integrity
• Best practices for managing study versions across battery updates

Overview

When you activate a study (move to Pilot or Active state), PEBL Hub creates a frozen snapshot - an immutable, archived copy of all test files used in your study. This frozen snapshot ensures that every participant in your study runs exactly the same version of each test, even if the PEBL test battery is updated months or years later.

Why Frozen Snapshots Matter

The Problem Without Freezing: Imagine you start data collection in January 2025 using the Iowa Gambling Task v2.1. In June 2025, PEBL releases Iowa Gambling Task v2.2 with improved graphics and slightly different timing. Without frozen snapshots:

• Participants in January get v2.1
• Participants in June get v2.2
• Your data is confounded - you can't tell if differences are due to your experimental manipulation or test version changes
• Your study is not reproducible - there's no way to rerun the exact version you used

The Solution With Frozen Snapshots: When you activate your study in January 2025, PEBL Hub:

1. Creates a frozen copy of Iowa Gambling Task v2.1
2. Locks it to your study
3. Serves this frozen version to all participants in your study
4. Keeps using v2.1 even when v2.2 is released
5. Notifies you about the update so you can make an informed decision about upgrading

Relationship to Regular Snapshots

PEBL Hub has two related but distinct snapshot systems:

Regular Snapshots (covered in Study Snapshots):

• Capture your study configuration: parameters, chains, settings
• Created automatically when you activate your study
• Track what you configured, not test file contents
• Can be downloaded as ZIP packages for reproducibility

Frozen Snapshots (this document):

• Capture the test files themselves: .pbl scripts, images, sounds, media
• Created at the same time as regular snapshots (when activating)
• Lock specific test versions to your study
• Cannot be changed once created (immutable)
• Served to all participants automatically

Together they provide complete reproducibility: Regular snapshots document what you configured (parameters, chains), while frozen snapshots preserve the exact test versions you ran.

How Frozen Snapshots Work

Automatic Creation

Frozen snapshots are created automatically when your study transitions to an active state:

1. Draft → Pilot: First frozen snapshot created
2. Pilot → Active: Reuses Pilot snapshot (if no changes) or creates new snapshot (if you modified configuration)
3. Disabled → Active (resuming): Reuses existing snapshot
4. Disabled → Draft → Pilot → Active (after major edits): Creates new snapshot

You don't have to do anything - the system handles it automatically.

What Gets Frozen

When a frozen snapshot is created, the system captures:

✅ Test files (.pbl scripts):

• Complete PEBL source code for each test
• Exact version from current battery

✅ Test resources:

• Images, graphics, stimuli
• Sound files, audio clips
• Media files specific to each test
• Data files, lookup tables

✅ Test metadata:

• Battery version (e.g., "3.0")
• PEBL compiler version
• File modification dates
• Checksums for integrity verification

What Doesn't Get Frozen

The frozen snapshot does not include:

❌ Your custom parameters:

• Loaded separately via API
• You can still adjust parameters after activation (though not recommended)
• Stored in regular snapshot configuration

❌ Your custom translations:

• Loaded separately
• Can be added/updated without breaking the frozen bundle

❌ Your test chains:

• Chain configuration stored separately
• Frozen bundle only contains the tests themselves

❌ Participant data:

• Data uploads go to separate storage
• Not part of snapshot system

Why separate? This separation allows you to make minor adjustments (parameters, translations) without breaking the frozen test files, while still protecting the core test code from changes.

Study States and Frozen Bundles

The frozen bundle system integrates with PEBL Hub's study lifecycle:

Draft State ✏️

• No frozen bundle - study is in development
• Tests are loaded from the live PEBL battery
• You're using the current version of each test
• Can test and experiment freely
• Good for setup and initial validation

Pilot State 🧪

• Frozen bundle created when you enter Pilot
• Your study now has a locked snapshot of all test files
• Participants in pilot get the frozen versions
• Tests won't change even if battery updates
• Good for validation with stable test versions

Active State ✅

• Uses frozen bundle from Pilot (or creates new one if needed)
• All real participants get identical test versions
• Protected from battery updates
• Ensures data consistency
• Required for reliable research

Disabled State ⏸

• Frozen bundle preserved but not served to participants
• URLs show "study paused"
• Bundle remains ready for resumption
• No changes to frozen files

Key Point: Once you activate (Pilot or Active), your tests are frozen. You're now protected from upstream changes to the PEBL battery.

Battery Updates & Notifications

What Happens When PEBL Updates Tests

PEBL Hub runs a daily check (typically at 2 AM) that:

1. Calculates checksums for all tests in the current battery
2. Compares to previous day's checksums
3. Detects which tests changed, were added, or were removed
4. Finds all active/pilot studies using changed tests
5. Creates notifications for affected researchers

Receiving Update Notifications

When a test you're using gets updated, you'll see:

Dashboard Notification: `` ⚠️ Battery Updates Available The following tests in your active studies have been updated in the PEBL test battery: Study: "Memory Study 2025" (Active) - Iowa Gambling Task: Updated in battery v3.1 - Stroop Task: Updated in battery v3.1 [Review & Upgrade] [Dismiss] ` **What this means**: - Your study is **still using the old frozen version** (good!) - A **new version** is available in the battery - You can **choose** whether to upgrade ### Reviewing Battery Updates Click **"Review & Upgrade"** to see detailed information: **Upgrade Review Page** shows: 1. **Current Snapshot Info**: - Which frozen snapshot your study currently uses - Battery version it was created with - Date created - Tests included 2. **Available Update**: - New battery version available - Which tests have changed - What changed (if known) 3. **Impact Assessment**: - How many participants have already completed (with old version) - Whether upgrade would affect data consistency 4. **Options**: - **Upgrade Now**: Create new snapshot with latest battery - **Keep Current Version**: Dismiss notification, continue with frozen version ### Making the Upgrade Decision **When to Upgrade**: ✅ **Study hasn't started collecting data yet**: - No participants completed - Safe to upgrade with no impact - Get latest bug fixes and improvements ✅ **Critical bug fix in test**: - PEBL team reports a serious issue - Bug affects data validity - Better to upgrade mid-study ✅ **Between cohorts or waves**: - Completed Wave 1 data collection - Upgrading before Wave 2 - Document version change in analysis ✅ **Enhancement doesn't affect core behavior**: - Cosmetic improvements - Performance optimizations - Better error messages - New translations added **When NOT to Upgrade**: ❌ **Mid-data collection**: - Already collected data from many participants - Would create inconsistency between early and late participants - Different participants would get different versions ❌ **Changes affect test behavior**: - Timing changes - Stimulus changes - Scoring changes - Task instructions modified ❌ **Close to publication**: - Data collection complete - Already documented current version in methods - Changing version would require methods revision ❌ **Unclear what changed**: - No changelog available - Unknown impact on behavior - Better to stay with known version ### How to Upgrade If you decide to upgrade: 1. **Click "Upgrade to Latest Battery"** button 2. **Study automatically pauses** (transitions to Disabled) 3. **New frozen snapshot created** with latest battery versions 4. **Study resumes** (returns to Active) with new snapshot 5. **Participants now get updated test versions** **What happens**: - ✅ Old frozen snapshot **preserved** (for participants who already completed) - ✅ New frozen snapshot created with updated tests - ✅ New snapshot becomes "current" for future participants - ✅ Snapshot history shows both versions - ✅ You can document version change dates **Important**: Document the upgrade! - Note the date of upgrade - Record how many participants completed with old version - Record how many completed with new version - Include in your data analysis and methods section ### How to Keep Current Version If you decide to stay with your current frozen version: 1. **Click "Keep Current Version"** button 2. **Notification dismissed** 3. **Study continues unchanged** with frozen snapshot 4. **No impact on participants** - they keep getting the old version 5. **Frozen bundle protects you** from the update **What happens**: - ✅ Your frozen snapshot continues to be served - ✅ Participants get consistent experience - ✅ You maintain data integrity - ✅ Can always upgrade later if needed **This is completely safe**: Your frozen bundle ensures participants will continue getting the exact same test version, regardless of what changes happen in the PEBL battery. ## Downloading Frozen Snapshots for Offline Use ### Running Frozen Snapshots with Native PEBL Frozen snapshots can be downloaded and imported into the **native PEBL launcher (version 2.3 or later)** for offline, in-person testing. **Why download frozen snapshots?**: - Run studies in-lab without internet dependency - Ensure participants in-lab get identical test versions to online participants - Maintain data collection continuity if internet is unavailable - Research+ accounts can sync in-lab data back to PEBL Hub automatically ### How to Download a Frozen Snapshot 1. Go to your study in PEBL Hub 2. Click **"📦 View Snapshot History"** 3. Find the frozen snapshot you want to download 4. Click **"📥 Download Frozen Bundle"** 5. Your browser downloads a ZIP file containing: - All test files (.pbl scripts) - All test resources (images, sounds, media) - Study configuration (parameters, chains, translations) - Token and URL information for data sync **Package structure:** ` StudyName_SNAP_a8f3e7b2.zip ├── battery/ # Frozen test files │ ├── iowa.pbl │ ├── stroop.pbl │ ├── images/ │ ├── sounds/ │ └── ... ├── config/ │ ├── study_config.json # Study configuration │ ├── parameters.json # Test parameters │ ├── chains.json # Test sequences │ └── translations/ └── README.txt # Import instructions ` ### Importing into Native PEBL 2.3 **Requirements:** - PEBL 2.3 or later installed on your computer - Downloaded frozen snapshot ZIP file **Steps:** 1. Extract the downloaded ZIP file to a folder 2. Open **PEBL Launcher 2.3** 3. Go to **File → Import Online Study** 4. Navigate to the extracted folder 5. Select studyconfig.json 6. PEBL imports the frozen snapshot and creates a local study **What gets imported:** - ✅ All test files from the frozen bundle - ✅ All test parameters - ✅ Complete test chains - ✅ Custom translations - ✅ Token ID and server URL (for data sync) ### Data Sync for Research+ Accounts **Research+ accounts** can run imported snapshots in-lab with automatic data synchronization: **How it works:** 1. Import frozen snapshot into PEBL 2.3 (as described above) 2. Run participants in-lab using native PEBL 3. When internet is available, PEBL automatically: - Uploads completed session data to PEBL Hub - Uses the study's token ID to associate data correctly - Syncs to the same dataserver as online participants **Benefits:** - Combine online and in-lab data collection in one study - All data appears in PEBL Hub data browser together - No manual data upload needed - Participants run identical frozen test versions whether online or in-lab **Sync requirements:** - Research+ account (standard accounts cannot sync) - Internet connection (periodic, not continuous) - Correct token and URL in imported snapshot (auto-included) **Sync status:** - PEBL 2.3 shows sync status indicator - Green: Data uploaded successfully - Yellow: Waiting for internet - Red: Sync error (check credentials) ### Use Cases **Hybrid online/in-lab studies:** - Recruit online participants → run frozen snapshot in browser - Recruit in-lab participants → run same frozen snapshot in PEBL 2.3 - All data collected under same study token - All participants get identical test versions **Internet reliability concerns:** - Download frozen snapshot as backup - If online study becomes inaccessible, continue in-lab - Maintain data collection schedule without interruption **Demonstration and training:** - Download frozen snapshot for offline demos - Show exact study experience without browser dependency - Train research assistants on study protocol offline ## Frozen Bundles and Reproducibility ### For Publication When publishing research using PEBL Hub: **In Your Methods Section**, document: ` We used the PEBL Test Battery (version 3.0) administered via PEBL Hub. Tests were frozen at study activation (January 15, 2025) using snapshot ID SNAP_a8f3e7b2c4d1. Tests administered: - Iowa Gambling Task (battery v3.0) - Stroop Task (battery v3.0) - Corsi Block Task (battery v3.0) All test parameters and frozen snapshot configuration are available at [DOI/URL]. The frozen snapshot bundle can be imported into PEBL Launcher 2.3 for exact replication. ` **Provide Reproducibility Materials**: 1. Download your frozen snapshot bundle (includes test files + configuration) 2. Download your configuration snapshot package (see [Study Snapshots](study-snapshots.md)) 3. Upload both to permanent repository (OSF, Zenodo, GitHub) 4. Get DOI for citation 5. Include: - Frozen bundle ZIP (complete study package) - snapshot.json (configuration documentation) - Snapshot ID for reference - Battery version used - Date of freezing - Import instructions for PEBL 2.3 **Why this matters**: - Other researchers can download and run your exact study - Works in browser (via PEBL Hub) or natively (via PEBL 2.3) - Can verify test versions are truly identical - Complete reproducibility: configuration AND test files - Meets highest reproducibility standards - Protects against "it worked in our version" problems ### For Replication Studies To replicate a published study that used frozen snapshots: 1. **Get the original snapshot ID** from paper's methods 2. **Contact original authors** for snapshot package 3. **Check battery version** - note if it matches current version 4. **Create new study** in PEBL Hub 5. **Configure to match** the published snapshot 6. **Document any differences** in battery versions **If battery version has changed**: - Your frozen snapshot will use current battery - Note version difference in methods - Compare test behavior if critical - Consider discussing impact in limitations ## Technical Details ### Frozen Bundle Storage For technical users and curious researchers: **Where frozen bundles are stored**: ` data/users/{USERNAME}/studies/{STUDY_TOKEN}/snapshots/{SNAP_ID}/ ├── battery_bundle.data # Frozen test files (binary) ├── battery_bundle.data.js # Loader script ├── battery_bundle.metadata.json # What's in the bundle ├── manifest.json # Snapshot information └── checksums.json # File integrity hashes ` **File sizes**: - Typical bundle: 50-200 MB (depends on number of tests) - Includes all test files for your study's tests only - Not the entire PEBL battery **Storage considerations**: - Frozen bundles count toward your storage quota - Old completed studies can be deleted to free space - Bundles are deduplicated by checksum (identical bundles reused) ### How Participants Receive Frozen Bundles **Web runtime** (browser-based PEBL): 1. Participant clicks your study URL 2. Runtime loads your study configuration 3. **Checks for frozen bundle** via API 4. If frozen bundle exists: - Downloads frozen bundle instead of current battery - Mounts bundle in virtual filesystem - Runs tests from frozen version 5. Participant completes tests using **your frozen versions** **Native PEBL** (downloadable snapshot package): 1. Download snapshot package ZIP 2. Extract files 3. Run tests with native PEBL 2 4. Tests are **exact copies** from frozen bundle 5. Upload data back to server (if configured) ### Checksums and Integrity Each frozen snapshot has: **File Checksums**: - SHA-256 hash of every file in bundle - Detects corruption or tampering - Verifies bundle integrity **Configuration Checksum**: - Hash of complete configuration (parameters, chains, translations) - Creates unique Snapshot ID - Enables deduplication **Battery Checksum**: - Hash of test file contents - Used to detect battery updates - Triggers notifications when tests change ## Best Practices ### 1. Activate Early in Development **Why**: Freezing early protects you from unexpected updates **Recommended workflow**: 1. Configure study in Draft 2. Test yourself thoroughly 3. **Transition to Pilot** (creates frozen snapshot) 4. Run pilot participants with frozen version 5. Transition to Active (reuses frozen snapshot) **Benefits**: - Pilot and real participants get identical versions - Protected from updates during pilot phase - Can pilot extensively without version drift ### 2. Document Your Frozen Snapshot **Record in your study notes**: - Date frozen snapshot was created - Battery version used - Snapshot ID - Any known issues with that battery version **Example note**: ` Frozen snapshot created: 2025-01-15 Battery version: 3.0 Snapshot ID: SNAP_a8f3e7b2c4d1 Tests: Iowa, Stroop, Corsi Notes: Using stable v3.0 release, no known issues ``

3. Check for Updates Regularly

Weekly or monthly:

• Review dashboard for battery update notifications
• Read changelogs for tests you're using
• Decide if updates are relevant

During active data collection:

• Generally do not upgrade unless critical bug
• Maintain version consistency across participants
• Document decision to stay with frozen version

4. Plan Upgrade Windows

If you must upgrade mid-study:

1. Choose upgrade timing strategically:

• Between recruitment waves
• At natural break points
• Not during active data collection

1. Document everything:

• Old version: battery v3.0, SNAPabc123, N=50 participants
• Upgrade date: March 15, 2025
• New version: battery v3.1, SNAP_def456, N=remaining participants

1. Analyze appropriately:

• Test for version effects in analysis
• Consider version as a covariate
• Report in methods and results
• Discuss in limitations if needed

5. Test After Upgrading

If you do upgrade:

1. Study auto-pauses during upgrade
2. Test the upgraded version yourself before resuming
3. Verify tests still work correctly
4. Check for behavioral changes
5. Only resume when confident

6. Keep Old Snapshots

Never delete old snapshots, even after upgrading:

Why preserve old snapshots:

• Some participants completed with old version
• Need for analysis and comparison
• Reproducibility requires old version info
• May need to reference in publications

Old snapshots don't hurt:

• Don't affect current study operation
• Preserved in snapshot history
• Available for download
• Prove study consistency

7. For Longitudinal Studies

Multiple measurement waves:

Option 1: Same frozen snapshot across waves

• Keep same study, same frozen snapshot
• All waves get identical test versions
• Maximum consistency

Option 2: Separate studies per wave

• Wave 1: Study A with Frozen Snapshot A (battery v3.0)
• Wave 2: Study B with Frozen Snapshot B (battery v3.1)
• Clearly document version differences
• Can still compare if tests stable

Recommendation: Use Option 1 (same study) unless you specifically want to incorporate battery improvements between waves.

8. Understand Your Protection

You are protected from:

• ✅ Automatic battery updates
• ✅ Test file changes
• ✅ Upstream modifications
• ✅ Breaking changes in tests
• ✅ Unintentional version drift

You are NOT protected from:

• ❌ Your own parameter changes
• ❌ Your own chain modifications
• ❌ Platform infrastructure updates (server, browsers)
• ❌ External dependencies (fonts, libraries)

Best defense: Transition to Pilot or Active early, and avoid making changes during data collection.

Common Questions

Can I create a frozen snapshot manually?

No, frozen snapshots are only created automatically during state transitions (Draft → Pilot or Pilot → Active).

Workaround: If you want to freeze current battery version:

1. Transition study to Pilot (if in Draft)
2. Frozen snapshot created automatically
3. Can immediately deactivate if not ready to collect data

What if I activated before frozen snapshots existed?

Older studies created before the frozen snapshot system was implemented are marked as legacy snapshots:

Legacy snapshots:

• Do not have frozen bundles
• Use the current PEBL battery (not frozen)
• Are susceptible to test updates
• May have been migrated to frozen bundles (check with administrator)

To convert legacy to frozen:

• Administrator can run migration script
• Creates frozen bundles retroactively
• Check with your platform administrator

Can I manually upgrade a specific test?

No, upgrades are all-or-nothing for the entire battery.

Why: Tests often depend on shared libraries and resources. Mixing versions could cause compatibility issues.

Workaround: If you need a specific test update:

1. Contact PEBL support
2. May be able to get individual test file
3. Upload as custom test
4. Use custom test instead of battery version

What happens if a test is removed from the battery?

Your frozen snapshot preserves it:

• Even if PEBL removes a test from future battery releases
• Your frozen bundle still contains it
• Participants can still run it
• You're protected from deprecation

For new studies: You won't be able to use the removed test (not in current battery).

Do frozen snapshots work with custom tests?

Yes, frozen snapshots include custom tests:

• Custom tests you uploaded are included in frozen bundle
• They're frozen just like battery tests
• Protected from changes to custom test files
• Even if you update your custom test, frozen version doesn't change

How do I know which version my participants got?

Check snapshot history:

1. Go to study management
2. Click "View Snapshot History"
3. See all snapshots created
4. Note dates and participant counts
5. Cross-reference with participant completion dates

In your data:

• Participant completion date < Upgrade date → Old version
• Participant completion date ≥ Upgrade date → New version

Can I have different frozen snapshots for different participants?

Not intentionally - all participants in an active study get the current snapshot.

But naturally this can happen if you upgrade:

• Participants before upgrade → Old snapshot
• Participants after upgrade → New snapshot
• Snapshot history preserves both versions

Best practice: Avoid upgrading during active data collection to ensure all participants get the same version.

Troubleshooting

Frozen bundle not created

Symptom: Study activated but no frozen snapshot in snapshot history

Possible causes:

• Database error during snapshot creation
• Storage quota exceeded
• Permission issues

Solutions:

1. Check snapshot history - may have been created despite error message
2. Try deactivating and reactivating
3. Check storage quota (may be full)
4. Contact administrator if persists

Tests not loading from frozen bundle

Symptom: Participants get errors, or tests load from current battery instead of frozen version

Possible causes:

• Frozen bundle file missing or corrupted
• API endpoint not accessible
• Study in wrong state

Solutions:

1. Verify study is in Pilot or Active state (not Draft or Disabled)
2. Check browser console for errors
3. Test yourself with "Try it out"
4. Contact administrator - may need to regenerate bundle

Battery update notification not received

Symptom: Test was updated but you didn't get notified

Possible causes:

• Notification system not configured
• Update check hasn't run yet (daily at 2 AM)
• Study already using latest version

Solutions:

1. Check dashboard manually for notifications
2. Wait until next daily check (runs at 2 AM)
3. Contact administrator if concerned

Can't dismiss notification

Symptom: Battery update notification won't go away

Solutions:

1. Click "Keep Current Version" explicitly (not just X)
2. May need to make a choice (upgrade or dismiss)
3. Refresh page
4. Contact administrator if persists

Upgrade failed mid-process

Symptom: Clicked upgrade but got error or study state unclear

Solutions:

1. Check study state - should be Active if upgrade succeeded
2. Check snapshot history - new snapshot should be listed
3. If in Disabled state - may need to manually resume
4. Contact administrator to verify snapshot created correctly

Related Topics

• Study Snapshots & Version Control - Complete snapshot system overview
• Managing Studies - Study lifecycle and state transitions
• Configuring Test Parameters - Setting up test parameters
• Uploading Custom Tests - Using your own test files

---

Need more help? Contact your platform administrator or PEBL support at support@pebl.com for technical assistance with frozen snapshots.

---

Related Topics

• Study Snapshots
• Managing Studies
• Configuring Test Parameters