Frequently Asked Questions

Quick Help

Can't find what you're looking for? Feel free to contact us or join our Discord community or Reddit community for direct support.

This page answers the most common questions we receive from researchers using Meadows. Use the search function or browse by category from the table of contents to your right.

Stimuli & Sampling

How do I define a specific subset of stimuli for each participant?

You can use the trial file option. This overrides other sampling methods like "Balanced".

Can I use words or sentences in non-latin scripts, such as Chinese or Hindi?

Yes. Create a Stimulus Set and upload a text file with your stimuli. Make sure your OS encodes the file as unicode.

How can I format multi-line text stimuli?

You can include line-break symbols in the text file to force wrapping. There are also settings in the Stimuli tab (multiline_text_scale) to choose how to scale stimuli with multiple lines.

Can I present separate stimuli for different tasks within one experiment?

You cannot load two different stimulus sets for one experiment. However, you can filter stimuli per task using the "Stimuli" tab settings: filter by name, filter by type (e.g., audio vs text), or use a trial file to specify which items appear in which task.

Can I optimize stimulus selection for pairs in the Timed Keyboard Response task?

Yes, the balanced sampling feature can optimize for pairs or triplets depending on the task parameters (e.g., stimuli_per_trial). See the Timed Key Response task documentation for configuration details.

Videos or images hang at "Loading..." for participants.

This can happen if the internet connection drops during the serial download of stimuli. Participants should be instructed to refresh the page (F5) to resume loading. For videos, ensure the format is supported (OGV is robust; h264 mp4 works on Chrome/Firefox but Safari has issues with certain request types).

"Unable to import" text stimuli CSV file.

The file must be Unicode/UTF-8 encoded and contain at least two columns (short form and long form). If only one version is needed, repeat the text in both columns.

Experiment Design & Task Configuration

How do I ensure the Multi-Arrangement (MA) task only has one trial?

You can set the "max session length" to a very low value (e.g., 0 minutes) or set the "Min required evidence weight" to 0.01. See the Multiple Arrangement task documentation for details on these parameters.

Can I collect captions for audio clips?

Yes. You can use the Label task for short text responses or the Stimulus Form task for longer text and slider ratings. Audio playback options (repeat, rewind) and display icons can be configured in the stimuli tab.

How do I make the arena/circle in the MA task larger?

The task ratio is fixed at 2:1. To maximize space: instruct participants to use full screen, use the "lateral display" option (stimuli start at sides), remove the "hint" text or make it smaller, and ensure image files do not have transparent padding. See the Multiple Arrangement task documentation for more configuration options.

Can I require a response for "Dynamic Form" questions?

For "input" (text) widgets, they require characters to be valid. For "select" (dropdown) widgets, the default "--Select--" is considered a valid HTML option, so users can technically skip it. You must check data post-hoc to disqualify those who did not change the selection. See the Dynamic Form documentation for more information about field types and requirements.

Can I insert images or GIFs into the instructions?

Yes, you must provide a full URL to a publicly hosted image (e.g., GitHub "raw" link) and paste this into the main text editor if you are using a Information Task. GIFs are also supported. Consider also the Slideshow task to display an instruction image by itself.

Can I rename an experiment?

This cannot be done in the interface, but we can rename it in the database if you contact us.

Billing

Can I pay in advance or top up the balance gradually?

Yes, it is possible to pay in advance, though most researchers prefer to pay after data collection is complete. Simply contact us and tell us how many minutes you expect to use.

Can I pay after my project has finished?

Yes, in fact this is how most projects proceed. Please set a reminder for yourself to contact us, set to activate on the date when you expect to finish. We do appreciate you sending us a brief email introducing yourself and your affiliation when your project starts.

How is the cost calculated?

Pricing is based on participant active minutes (e.g., £0.10 per minute). The first 120 minutes are free. You are charged for time participants are active, excluding breaks longer than 1 minute.

Does the "120 free" refer to participants or minutes?

It refers to participant-minutes (e.g., 20 participants x 10 minutes = 200 minutes; minus 120 free, means you'd be charged 80 minutes; £8 / $10.40 / €9.60).

Do you charge for unfinished participants?

Yes, costs are incurred for all participant activity, even if they drop out, though sometimes discounts can be applied for failed pilots or bugs.

Can I get an invoice for a specific amount or split between grants?

Yes, invoices can be split or adjusted upon request.

How do I pay?

When data collection is done, request an invoice. It will be emailed with a link to a payment portal to pay by card. Bank transfers are also possible; please tell us which currency you'll be paying in (as the bank details vary by currency).

Can you send me a quote?

Of course, please email us and tell us the amount and who we should address the quote to.

Data & Analysis

How can I visualize/analyze 3AFC (odd-one-out) data?

There is no off-the-shelf visualization, but we can help you write a script to transform Meadows output into formats compatible with libraries like SPoSE. See the Triplet Choice task documentation for examples of analyzing odd-one-out data.

How is the RDM vector in the output formatted?

It is the upper triangular vector of the matrix (row-major order). It can be converted to a square matrix using squareform() in Matlab or scipy.spatial.distance.squareform in Python. The order corresponds to the stimuli variable in the data file. See the Multiple Arrangement task documentation for examples of working with RDM data.

How do I extract trial durations for MA trials?

Look at the "log" events in the JSON/events file. Subtract the "started" timestamp from the "finished" timestamp for each trial. Timestamps are in milliseconds (epoch).

Why are there NaNs in the group RDM file?

If a participant has not finished the experiment, or if the "max duration" was reached before all pairs were judged, the RDM will contain NaNs.

Is there a toolbox to aggregate Card Sorting data where users define their own piles?

There is no specific toolbox function yet. A recommended approach is to estimate a co-occurrence RDM (how often items appear in the same pile) or use 1-hot encoding. See the Card Sorting task documentation for data analysis examples.

How do I interpret the coordinates in the Multi-Arrangement task?

Coordinates are relative to the arena size. (0,0) is the center, and +1 or -1 represents the edges (e.g., right edge or top). See the Multiple Arrangement task documentation for details on the coordinate system.

How do I convert JSON tree data to CSV?

There is no standard way due to complexity (mixed structures). You must load the JSON in Python/R and extract specific fields to a table. See the tree structure and downloads documentation for more information on data formats.

Participant Management

Batch creation fails with "E11000 duplicate key error".

This occurs when the system fails to generate a unique participant ID (adjective-noun combo) for a large batch (1000+ slots). You can retry or make a smaller batch. A very large batch is only recommended for experiments where you've already collected data and checked it.

Participants on Prolific are "Awaiting Review" but have no completion code.

This usually means they did not finish the experiment. You can verify this by checking if they are marked as "finished" in the download data toggle or reviewing the metadata file.

Experiment link says "currently not recruiting" even though slots appear open.

This may occur if a participant is missing required parameters set in the batch configuration. This could happen when something goes wrong when copying the URL, or when the recruitment platform makes a change. Also check if the link you've copied is for a specific (other) batch.

Participant claims to have finished but data is missing.

If the last task is "General Information" (e.g., a debrief), the participant must click the final "I understand" or "Finish" button. If they close the browser before this, they are not marked "finished." Data is still accessible via individual JSON downloads.

How do I link Meadows data to Prolific or MTurk IDs?

Use the "Prolific" template in the New Batch form. Alternatively, add a Dynamic Form task asking for the ID, or use URL entry variables (e.g., ?PROLIFIC_PID={{%PROLIFIC_PID%}}) which are stored in the metadata file. See the Prolific recruitment documentation for more details.

Can I redirect participants to another URL (e.g., Qualtrics/REDCap) after the task?

Yes. You can use the "Completion Link" feature. The domain must be whitelisted by the admin. To pass parameters (like IDs) back, you may need to use {{%PROLIFIC_PID%}} or similar templates in the URL. For REDCap, an exitUrl parameter was implemented. See the REDCap recruitment documentation for more details.

Can I prevent participants from using mobile phones?

Meadows does not strictly block phones, but you can check the screen resolution in the data logs (widths < 800px are often phones). You can also include a Card Measure task to measure physical screen size.

Can I reactivate expired batches to use remaining slots?

No. Once a batch expires, unused slots are deleted from the database. You must create a new batch.

Can a participant resume a session if they close the browser?

If they are within the timeout window that you've configured (e.g., 24h) and cookies are active, they can resume. If they are logged out, they need to log back in (if they have an account). Anonymous users cannot log back in once the session is lost. We can override this on an ad-hoc basis on request.

Is there a way to allow participants to retake the experiment multiple times?

A "take part again" button can be enabled on the thank-you page, which acts like a logout and redirect loop. When creating a batch, check the box ask to take part again.

Miscellaneous

How should I cite you in my paper or conference submission?

We'd very much appreciate it if you could include a url, for instance https://meadows-research.com, and/or the url of the documentation page for the tasks you used, e.g. https://docs.meadows-research.com/tasks/multiple-arrangement/

Where is your data stored?

In London, UK.