Importing Time Series Data from csv-files

Alexander Hagmann
A free video tutorial from Alexander Hagmann
Data Scientist | Finance Professional | Entrepreneur
4.6 instructor rating • 6 courses • 26,962 students

Learn more from the full course

The Complete Pandas Bootcamp 2020: Data Science with Python

Pandas fully explained | 150+ Exercises | Must-have skills for Machine Learning & Finance | + Scikit-Learn and Seaborn

31:50:30 of on-demand video • Updated September 2020

  • Bring your Data Handling & Data Analysis skills to an outstanding level.
  • Learn and practice all relevant Pandas methods and workflows with Real-World Datasets
  • Learn Pandas based on NEW Version 1.0 (the days of versions 0.x are over)
  • Import, clean, and merge messy Data and prepare Data for Machine Learning
  • Master a complete Machine Learning Project A-Z with Pandas, Scikit-Learn, and Seaborn
  • Analyze, visualize, and understand your Data with Pandas, Matplotlib, and Seaborn
  • Practice and master your Pandas skills with Quizzes, 150+ Exercises, and Comprehensive Projects
  • Import Financial/Stock Data from Web Sources and analyze them with Pandas
  • Learn and master the most important Pandas workflows for Finance
  • Learn how to best transition from Versions 0.X to new Version 1.0
  • Learn the Basics of Pandas and Numpy Coding (Appendix)
  • Learn and master important Statistical Concepts with scipy
English [Auto] In this video we are going to impart our very first time series data and then our fights. We have a CSG fight called temperature or temp and it contains temperature information for New York and Los Angeles for the yes. 2013 to 2016 and in a very first step. Is actually no difference between importing longtime serious data and time serious data from CSP fires. So first of all we import pandas and then we are using the method. Read CSB. So we are importing our CSP fire temperature our temp and we actually store our new data frame then in the web temperature and let's have a very first look at our data frame so we we can see the first five rows and actually we have a sweet column. So we have the column date time we have L.A. for Los Angeles and we have New York and apparently here as well use of our data frame we have actually the temperature at this specific time in these specific city and the temperature is actually in degrees Celsius. So for example here we have January eleven point seven in L.A. and minus one point one degrees Celsius in New York. So let's get some meta information with the info method and on the left hand side we have a range index and we have actually thirty five thousand sixty four rows. So we have over thirty five thousand different the timestamps here and with actually three columns. So we have the date time column and obviously here we have string data type. So here we have object and this is a clear indication that to we to the strings in the column then we have the column in Los Angeles service the flow data types. So these are your degrees Celsius values and therefore float the data type. Makes sense and also here for the New York column we have also a float data type. So the first problem is here that in the daytime column we actually do not have a specific date time data type. So we have a string that I type and this might not be really helpful and therefore on a first step we can try to transform data information that this actually starts here and strings into a date time data type of format and we can do this immediately when we import the data frame so they read CSP method provide some parameters that helps to import the date time information and let's go inside and if we can see that we have the pyramid up past dates and by default it set to false but let's have a look here. So apostates and Tippett can actually pass column labels for columns that we want to transform from string data type to a date time data type and we have to do this as a list and obviously our column that we want to transform a the daytime column. So we pass the time and we can also look at the documentation for the past dates power Mehta so weird us. And for example here we can pass the Boolean value true and then pan out tries to pass them the index into a date time format. But obviously here we have a Range Index. So we do not have any date time information on the index or we can pass a list of integers or names so we can pass either a list with the index position of the columns or the column labels and this is exactly what we do what we do here we pass that the column label daytime and let's see what we got here. So we actually overwriting our variable temperature and let's have a look here again at the Inform method and here we can see now that the data type changed from object to a daytime 64 ante and spat brackets we have an annual seconds. So the position is nanoseconds if we want to have this. So now we can also update you on the view on the first five rows. And now we have had the column date time with the data type date times 64. And we can also slice your fun element in our daytime column and we can select here the very first element myth of the ILO operator. And here we have a so-called timestamp and a time stamp is actually a single point in time. So for example here we have the year 2013 we have a 10 year Ria and we have it and you read the first and then also we have information on the time. So we have at midnight here. So these are the hours. Then we have minutes and then seconds. And as you can see above our data frame provides hourly temperature information. So the increment this one hour and we're starting here with the January the 1st at midnight then our next timestamp is this third January the 1st. 1:00 a.m. Then we have 2:00 a.m. 3:00 and so on and we can also check here the data type off the time stamp of that type of method and this is here a panda's timestamp. So the timestamp is a specific Panda's data type to store daytime information. And whenever we have daytime information like this it makes sense to have the daytime as the index of our data frame. And by doing so this gives us a lot of additional functionality. So we can already do this when we import our data obviously read CSA method and we've already seen that that is that the parameter index calls and Tippecanoe past the column labor for the column that you want to have as index and in this case it's your day time. So this is actually a very common workflow with time series that when we import the data we first of all pass the columns that contain the daytime information with the parent to pass dates and then being can to these columns as the index. So let's do this here and be all right. Our variable temperature and have again a look at the first five rows and now we can see here that we have our daytime column. Now as the index and we can also check this with the informal method so we can see here that we have a so-called daytime index and we have 35000 entries and these range from January the 1st 2013 midnight until December thirty first of 2016 11:00 p. m.. So we feel all the temperature information for New York and Los Angeles for the year 2000 and 30 into 2016. And we can also call you to the index attribute to further analyze our new index. And here we have our daytime index until we have all of the timestamps. So a daytime index is actually the collection of many timestamps and we can see at the bottom that we have the data type daytime 64 and index label we have daytime and we have thirty five thousand entries and we can also select a single element in our index so let's select the very first element and this is here the very first timestamp. So January of the 1st 2013 at midnight. All right. This were the first steps with time series and I hope to see you also in the next video by.