Data Cleaning

Data Cleaning

Introduction

Following data collection, it's essential to ensure the validity of the collected data and address any instances where participants may have completed the questionnaire without due attention. To tackle this issue, we've introduced a feature known as Data Cleaning. This functionality incorporates various tools designed to assist you in "cleaning" your data prior to analysis.

Response Time

This feature is part of the data cleaning toolkit and enables the calculation of the minimum and maximum time a participant should take to complete the questionnaire, based on the collected session data. Participants falling outside of this time range will be flagged as either "Too Slow" if they took longer than the mean time of the entire panel, or "Too Fast" if they completed the questionnaire too quickly compared to the panel's mean time.

When this functionality is enabled, the system suggests minimum and maximum time thresholds, calculated based on the average time plus or minus two times the standard deviation. You have the flexibility to adjust these thresholds within the Start Time and End Time fields. Clicking "Flag" will identify sessions that were completed either too slowly or too quickly, marking them accordingly in the session overview table under the "Data Cleaning" column.

From there, you can choose to either remove or disable flagged sessions. If you opt to remove the data, sessions from those participants will be permanently deleted from the system and cannot be retrieved. If you decide to disable these sessions, they will be excluded from analysis, auto-reports, and raw data. However, you retain the flexibility to reactivate them as needed.

Question Filter

A question filter is a tool that allows the users to apply specific criteria to data fields in order to detect abnormalities, errors or incomplete responses. Instead of going through the entire dataset manually, the user can filter the records that meet certain criteria. 

When this functionality is enabled the user can select the question(s), only the category questions are available, and the criteria it should pass. If there are more question filters to be applied there is the possibility to change the logic AND/OR. Once the filters are done the user can flag the sessions. The flagged sessions will be identified by the Question Filter tag.
 

From there, you can choose either to remove or disable the flagged sessions. If you opt to remove the data, sessions from those participants will be permanently deleted from the system and cannot be retrieved. If you decide to disable these sessions, they will be excluded from analysis, auto-reports, and raw data. However, you retain the flexibility to reactivate them as needed.

It is as well possible to use both the data cleaning tools simultaneously. 


Idea
To bulk delete/disable the flagged sessions the user is able to filter via the drop down on either the Question or Response filter and select the sessions at once. 


    • Related Articles

    • Quality Scoring / Rating / Grading

      Introduction This template lets panelists assign a numerical score to each sample, and all samples are displayed at once. Template Description The template starts with a screen which can be used for welcoming panellists or asking additional questions ...

    • Time Intensity Analysis

      Purpose Time Intensity tests are designed to measure the temporal evolution of a single attribute. Results are plotted as time-intensity curves and key parameters of the curves are calculated. Results can be averaged per panellist or per product. ...

    • Quality Index Analysis

      Available from version: 5.4.4 Purpose The Quality Index is an ANOVA based analysis, with the idea based on the paper by Verhoef (2015) for the purpose of measuring the reliability of univariate sensory descriptive data. Data format profiling.xlsx The ...

    • Means over Time

      Available from version: 5.3.1 Purpose The means over time analysis is a helpful tool that can assist you in seeing how a product changes over time. This type of analysis is often useful for quality control, and it consists of a line chart where the ...

    • How Can I Analyse My Data?

      In EyeQuestion there are multiple options to analyze the project data. When you select the Data tab in your project you will find a dropdown menu Analysis: Auto Reports Via the option for Auto reports EyeQuestion will analyze the data and create the ...