Working with Data

Working with Data InfoGuide

This guide is to learn basic skills in data cleaning and data wrangling including how to use a computer. 

• Using a Computer - Tips for understanding and using your computer efficiently
  • The Internet - Tips for downloading and staying safe
  • File Formats & Conversion - What to do with files you've downloaded
  • Working with Files - How to navigate your file system to find, rename, and manage files
• File Management - Best Practices in file management and organization
  • Open Science Framework - Another Option to manage and share your files
  • Git / GitHub - Learn how to more easily backup and share your text files
  • Big Data - Working with large datasets requires different methods
• Data Cleaning - Why and how to go about cleaning
  • OpenRefine - A free open source platform for data cleaning and management
  • Spreadsheets - Use Excel or other spreadsheets for data management, data cleaning, and summarizing data
  • Web Scraping - How to extract data from the web
• Coding - General programming concepts and languages
  • Command Line / Shell - Learn about the command line including Bash and DOS
  • Regular Expressions - The common language of Find-and-Replace
  • SQL & Databases - Relational Database concepts and the common language of Databases
• Get Help - If you need assistance with any of these issues, contact the Digital Scholarship Center

Fundamentals

• Essentials of Computing with Data / Slides (D-Lab, UC Berkeley)
  • File Organization, Command Prompts, Storage Formats, Storage Sizes, Servers, Text Editors, Scripting
• Doing Digital Scholarship (SSRC Labs)
  • Digital Projects, Publishing, Close Reading, Mapping, Visualization

More Topics

• Introduction to Computational Tools for Social Science
  • Unix, R, APIs, Webscraping, MTurk, Phython, Text Analysis, Machine Learning
• Software Carpentry
  • Unix, Version Control (e.g., Git), Mercurial, Databases & SQL, Python, R, MATLAB, Automation
• Data Carpentry
  • Spreadsheets, OpenRefine, R, Python, SQL, Project Organization and Management
• School of Data Online Courses
  • Spreadsheets, OpenRefine, Presenting Data, SQL, Web Scraping, Geocoding, QGIS, Open Data Kit, Timemapper.js

• UK Data Service
  • Data Skills Modules: Survey Data/Sampling, Longitudinal Data, Aggregate Data

• Institute for Quantitative Social Science (Harvard)
• D-Lab (UC Berkeley)
• Social Science Computing Cooperative  (U Wisconsin Madison)
• DataLab (Tufts)
• Research Data Services + Sciences / StatLab (UVA)