Clean Data - Data Science Strategies for Tackling Dirty Data by Megan Squire

By Megan Squire

Key Features

  • Grow your info technology services by way of filling your toolbox with confirmed thoughts for a large choice of cleansing challenges
  • Familiarize your self with the the most important facts cleansing approaches, and proportion your personal fresh info units with others
  • Complete real-world initiatives utilizing info from Twitter and Stack Overflow

Book Description

Is a lot of a while spent doing tedious projects equivalent to cleansing soiled information, accounting for misplaced facts, and getting ready facts for use by way of others? if this is the case, then having the suitable instruments makes a serious distinction, and may be a very good funding as you develop your info technology expertise.

The booklet begins by way of highlighting the significance of information cleansing in information technology, and may enable you obtain rewards from reforming your cleansing technique. subsequent, you are going to cement your wisdom of the fundamental ideas that the remainder of the ebook is determined by: dossier codecs, information kinds, and personality encodings. additionally, you will methods to extract and fresh information kept in RDBMS, internet records, and PDF records, via useful examples.

At the top of the booklet, you'll be given an opportunity to take on a number of real-world projects.

What you are going to learn

  • Understand the position of information cleansing within the total information technological know-how process
  • Learn the fundamentals of dossier codecs, info kinds, and personality encodings to wash facts properly
  • Master serious gains of the spreadsheet and textual content editor for organizing and manipulating data
  • Convert info from one universal layout to a different, together with JSON, CSV, and a few special-purpose formats
  • Implement 3 various recommendations for parsing and cleansing facts present in HTML documents at the Web
  • Reveal the mysteries of PDF records and tips on how to pull out simply the knowledge you want
  • Develop more than a few ideas for detecting and cleansing undesirable information saved in an RDBMS
  • Create your personal fresh facts units that may be packaged, authorized, and shared with others
  • Use the instruments from this publication to accomplish real-world initiatives utilizing info from Twitter and Stack Overflow

About the Author

Megan Squire is a professor of computing sciences at Elon collage. She has been gathering and cleansing soiled facts for 2 many years. She is additionally the chief of FLOSSmole.org, a examine undertaking to gather info and study it that allows you to find out how loose, libre, and open resource software program is made.

Table of Contents

  1. Why do you want fresh Data?
  2. Fundamentals codecs, kinds, and Encodings
  3. Workhorses of fresh information Spreadsheets and textual content Editors
  4. Speaking the Lingua Franca facts Conversions
  5. Collecting and cleansing facts from the Web
  6. Cleaning information in Pdf Files
  7. RDBMS cleansing Techniques
  8. Best Practices for Sharing Your fresh Data
  9. Stack Overflow Project
  10. Twitter Project

Show description

Read Online or Download Clean Data - Data Science Strategies for Tackling Dirty Data PDF

Similar python books

Getting Started with Beautiful Soup

The right way to extract details from web pages utilizing appealing Soup and the Python urllib2 module. This sensible, hands-on advisor covers every thing you want to recognize to get a head commence in site scraping.

Programming Python (4th Edition)

If you've mastered Python's basics, you're able to begin utilizing it to get genuine paintings performed. Programming Python will express you ways, with in-depth tutorials at the language's fundamental software domain names: process management, GUIs, and the net. You'll additionally discover how Python is utilized in databases, networking, front-end scripting layers, textual content processing, and extra.

Learning BeagleBone Python Programming

BeagleBone is a barebone machine that may be configured and customised for various functions and is nearly part the cost of a typical computing device. This booklet will hide the fundamentals of the way BeagleBone Black's interface subsystems paintings, and will be managed utilizing well known Python libraries for BeagleBone Black.

Python Essentials - A Rapid Guide to the Fundamental Features of Python

Key FeaturesLearn the best way to write powerful Python codeGet up-to-speed with crucial facts constructions in Python, utilizing sequences, units, and mappingsExplore Python's extraordinary variety of positive aspects and become aware of why it really is essentially the most well known programming languages within the worldBook DescriptionPython necessities offers an essential travel of the main serious good points of Python.

Additional resources for Clean Data - Data Science Strategies for Tackling Dirty Data

Sample text

An Excel expert will be unfazed if asked to parse out the pieces of a few hundred street addresses or sort a few thousand book titles into categories. No statistician or programmer is going to blink if asked to count the character frequency in a word list. But when string manipulation turns into "extract the source code found embedded in 90 million e-mail messages written in Russian" or "calculate the lexical diversity of the entire contents of the Stack Overflow website", things get a lot more interesting.

Depending on what kind of storage system the integers are stored in, for example, in a Database Management System (DBMS), we may also have additional information about how large an integer can be stored as well as whether it is allowed to be signed (positive or negative values) or only unsigned (all positive values). Numbers with decimals In our data cleaning work, numbers with a fractional component—such as prices, averages, measurements, and the like—are typically expressed using a decimal point (rather than a numerator/denominator).

But then again, maybe we can be more strategic about this. Whether we need to have zero values in our raw data actually depends on what tool we are using to create the graph and what type of graph it is, for example, Google Spreadsheets will build a line or bar graph that can automatically detect that there are missing dates on the x axis and will fill in zero values even if they are not given in the initial dataset. In our data, these zero values would be the mysterious missing dates from most of 1998.

Download PDF sample

Rated 4.35 of 5 – based on 33 votes