QUANTitative Analysis & Statistics: FAQs

1. Your statistical software -- Reduces possible errors
2. Any statistical software -- Includes labels and other metadata
   • SPSS (.sav), Stata (.dta), SAS (.sas7bdat), R(.rdata), Jamovi (.omv), etc.
3. Fixed Format with a Setup File **
   • Setup Script: SPSS (.sps), Stata (.do and .dct), SAS (.sas), R(.R)
   • Data File: Text or ASCII file (.dat or .txt) -- will not have variable names at the top
4. Spreadsheet format
   • "Delimited" files: Comma Separated Values (.csv) or Tab Separated Values (.tsv or .tab)
   • Excel or the equivalent (e.g., .xls, .xlsx)
   • May be labeled ASCII or given as .dat or .txt, (but without a Setup Script file as in #3)
5. Other formats:
   • Structured Text:  JSON (.json), XML (.xml)
   • Many more!

** Requires a Setup Script, Data file, and access to the designated software. Look for instructions in the Script file explaining how to specify the location of the datafile and "run" the script. For assistance, talk to your local Data Librarian.

Especially if your data file is in a format different from your software, there can be problems when you first open it. Check for these common issues. 

• String fields may get truncated (lose text) if the field size is not guessed correctly by the software, or look garbled if the wrong decoding is used (e.g., ASCII vs UTF-8). Check fields that may have lots of text (e.g., open-ended answers) for complete content. You will have to re-import, specifying the correct settings.
• Variable Names may be changed to be valid. Spaces will be removed or converted to underscores. Alternatively, the name could be changed to something generic, like V1 or X. Specify names during import, or rename afterwards.
• Special Field Types
  • Date fields may not be parsed or interpreted correctly. Check missing values and spot-check the year against the original source.
  • Numeric fields with non-number characters (including $, %, &, or .) may be imported as strings, or may cause the entire value to be set as missing.
  • If necessary, import these fields as strings to start and use the appropriate software functions to properly format the data at a later point if needed. Set up primary data properly to avoid these issues.

• Variables not separated properly into columns. This may only affect a few observations, but leads to obviously incorrect values in some variables. One observation might also be split across multiple rows. Check frequency tables for excess missing values or unexplainable values. In most cases, you will have to re-import with the proper settings to fix this.
• Observations might disappear due to file corruption, or extra rows (with mostly missing values) could be included due to headers, notes, or summaries. Confirm that the number of observations matches expectations. If there are extra rows, they can be deleted. If the file is corrupted, you may have to re-obtain it.

Depending on requirements related to your field and the type of project you are completing (e.g., dissertation), analyses could include: a) Descriptive Statistics, b) Bivariate Inferential Statistics, c) Standard multivariate analyses such as ANOVA and Multiple Regression (Linear or Logistic), or d) other Modeling (including GLM, Mixed Models, or SEM). This will influence how much data is needed, necessary software, and the criteria the data must meet.

It is useful to be able to conceptualize the ultimate data table you will need and even create it with some example data. The research questions should define  a) the unit of analysis, b) the population (observations), and the constructs (variables).

If you do not have research questions, that usually means you are expected to come up with a research question you can answer with the data you have. If so, consider which of the topics addressed by the data interest you. You will want to choose two topics and look for relationships between the values or group differences

Is the unit of observation a person, a place, or something else? Is it the same as the unit of analysis? Find the variable(s) that contain unique identifiers and determine their meaning. If you have multiple data tables/files, is the unit of observation the same and how will the rows match up--you can aggregate your data to make the units bigger, but not the opposite. Are there repeated measures, multi-level, time series, or panel data? If so, you might need to reshape the data or use special analyses.

How much of the data is actually from your target population? Both primary and secondary data may have missing data or extra observations that are not relevant, but it is important to end up with a sufficient number of observations. Also, what type of sampling method was used, if any? Were snowball techniques used, or complex sampling methods used to select the samples? Do you even want to generalize from your sample to the population, and how much work will it be if so? 

The data source gives a good idea of the data quality and how much time and effort will be needed to process it. Large, reputable, data collectors will have clean data with lots of documentation. Smaller sources might need more checking for data errors and may not have all the information necessary. If you have collected you own data, you may not need to narrow down the variables needed, but there are various cleaning steps you will need to take depending on the mode of collection (e.g., mail vs online), population (e.g., paid vs volunteer), and other factors.

Is it on paper, in Excel, a file in SPSS format, or what? Is there just one file/table, or multiple? The data format affects how quickly you will be able to use the data in your statistical software. Recent versions of all major software packages can directly open files from each other (and Qualtrics can export to SPSS format). But conversions always require checking. Older or smaller data may even come in a text format like CSV or fixed format that are more difficult to open and do not already have labels.