《利用Python进行数据分析》笔记---第5章pandas入门
写在前面的话:实例中的所有数据都是在GitHub上下载的,打包下载即可。地址是:http://github.com/pydata/pydata-book还有一定要说明的:我使用的是Python2.7,书中的代码有一些有错误,我使用自己的2.7版本调通。# coding: utf-8from pandas importSeries, DataFrameimport pandas as p
·
写在前面的话:
实例中的所有数据都是在GitHub上下载的,打包下载即可。
地址是:http://github.com/pydata/pydata-book
还有一定要说明的:
我使用的是Python2.7,书中的代码有一些有错误,我使用自己的2.7版本调通。
# coding: utf-8
from pandas import Series, DataFrame
import pandas as pd
import numpy as np
obj = Series([4,7,-9,7])
obj
obj.values
obj.index
obj2 = Series([4,5,6,3],index = ['a','b','e','c'])
obj2
obj2.index
obj2['a']
obj2[['a','b','c']]
obj2[obj2>4]
obj2 * 2
np.exp(obj2)
'b' in obj2
'r' in obj2
sdata = {'Ohio':35000, 'Texas':71000, 'Oregon':16000, 'Utah':5000}
obj3 = Series(sdata)
obj3
states = ['California','Ohio','Oregon','Texas']
obj4 = Series(sdata, index=states)
obj4
pd.isnull(obj4)
pd.notnull(obj4)
obj4.isnull()
obj3 + obj4
obj4.name = 'population'
obj4.index.name = 'state'
obj4
obj.index = ['Bob', 'Steve', 'Jeff', 'Ryan']
obj
data = {'state':['Ohio','Ohio','Ohio','Nevada','Nevada'],'year':[2000,2001,2002,2001,2002],'pop':[1.5,1.7,3.6,2.4,2.9]}
frame = DataFrame(data)
frame
DataFrame(data,columns=['year','states','pop'])
frame2 = DataFrame(data, columns=['year','state','pop','debt'],index=['one','two','three','four','five'])
frame2
frame2.columns
frame2['state']
frame2.year
frame2.ix['three']
frame2['debt'] = 16.5
frame2
frame2['debt'] = np.arange(5.)
frame2
val = Series([-1.2,-1.5,-1.7],index=['two','four','five'])
frame2['debt'] = val
frame2
frame2['eastern'] = frame2.state == 'Ohio'
frame2
del frame2['eastern']
frame2.columns
pop = {'Nevada':{2001:2.4,2002:2.9},'Ohio':{2000:1.5,2001:1.7,2002:3.6}}
frame3 = DataFrame(pop)
frame3
frame3.T
DataFrame(pop,index=[2001,2002,2003])
pdata = {'Ohio':frame3.Ohio[:-1],'Nevada':frame3.Nevada[:2]}
DataFrame(pdata)
frame3.index.name = 'year';frame3.columns.name = 'state'
frame3
frame3.values
frame2.values
obj = Series(range(3), index=['a','b','c'])
index = obj.index
index
index[1:]
index[1] = 'd'
index = pd.Index(np.arange(3))
obj2 = Series([1.5,-2.5,0],index=index)
obj2.index is index
frame3
'Ohio' in frame3.columns
2003 in frame3.index
obj = Series([4.5,7.2,-5.3,3.6],index=['d','b','a','c'])
obj
obj2 = obj.reindex(['a','b','c','d','e'])
obj2
obj.reindex(['a','b','c','d','e'],fill_value=0)
obj3 = Series(['bule','purple','yellow'],index=[0,2,4])
obj3.reindex(range(9),method='ffill')
frame = DataFrame(np.arange(9).reshape((3,3)), index=['a','c','d'],columns=['Ohio','Texas','California'])
frame
frame2 = frame.reindex(['a','b','c','d'])
frame2
states = ['Texas','Utah','California']
frame.reindex(columns=states)
frame.reindex(index=['a','b','c','d'],method='ffill',columns=states)
frame.ix[['a','b','c','d'],states]
obj = Series(np.arange(5.),index=['a','b','c','d','e'])
new_obj = obj.drop('c')
new_obj
obj.drop(['d','c'])
data = DataFrame(np.arange(16).reshape((4,4)),index=['Ohio','Colorado','Utah','New York'],columns=['one','two','three','four'])
data.drop(['Colorado','Ohio'])
data.drop('two',axis=1)
data.drop(['two','four'],axis=1)
obj = Series(np.arange(4.),index=['a','b','c','d'])
obj['b']
obj[1]
obj[2:4]
obj[['b','a','d']]
obj[{1,3}]
obj[obj < 2]
obj['b':'c']
obj['b':'c'] = 5
obj
data = DataFrame(np.arange(16).reshape((4,4)),index=['Ohio','Colorado','Utah','New York'],columns=['one','two','three','four'])
data
data['two']
data[['three','one']]
data[:2]
data[data['three'] > 5]
data < 5
data[data < 5] = 0
data
data.ix['Colorado',['two','three']]
data.ix[['Colorado','Utah'],[3,0,1]]
data.ix[2]
data.ix[:'Utah','two']
data.ix[data.three > 5, :4]
s1 = Series([7.3,-2.5,3.4,1.5],index=['a','c','d','e'])
s2 = Series([-2.1,3.6,-1.5,4,3.1], index=['a','c','e','f','g'])
s1
s2
s1 + s2
df1 = DataFrame(np.arange(9.).reshape((3,3)),columns=list('bcd'),index=['Ohio','Texas','Colorado'])
df2 = DataFrame(np.arange(12.).reshape((4,3)),columns=list('bde'),index=['Utah','Ohio','Texas','Oregon'])
df1
df2
df1 + df2
df1 = DataFrame(np.arange(12.).reshape((3,4)),columns=list('abcd'))
df2 = DataFrame(np.arange(20.).reshape((4,5)),columns=list('abcde'))
df1
df2
df1 + df2
df1.add(df2, fill_value=0)
df1.reindex(columns=df2.columns,fill_value=0)
arr = np.arange(12.).reshape((3,4))
arr
arr[0]
arr - arr[0]
frame = DataFrame(np.arange(12.).reshape((4,3)),columns=list('bde'),index=['Utah','Ohio','Texas','Oregon'])
series = frame.ix[0]
frame
series
frame - series
series2 = Series(range(3),index=['b','e','f'])
frame + series2
series3 = frame['d']
frame
series3
frame.sub(series3,axis=0)
frame = DataFrame(np.random.randn(4,3),columns=list('bde'),index=['Utah','Ohio','Texas','Oregon'])
frame
np.abs(frame)
f = lambda x:x.max() - x.min()
frame.apply(f)
frame.apply(f,axis=1)
def f(x):
return Series([x.min(),x.max()],index=['min','max'])
frame.apply(f)
format = lambda x: '%.2f' % x
frame.applymap(format)
frame['e'].map(format)
obj = Series(range(4),index=['d','a','b','c'])
obj.sort_index()
frame = DataFrame(np.arange(8).reshape((2,4)),index=['three','one'],columns=['d','a','b','c'])
frame.sort_index()
frame.sort_index(axis=1)
frame.sort_index(axis=1,ascending=False)
obj = Series([4,7,-1,3])
obj.order()
obj = Series([4,np.nan,7,-1,np.nan,3])
obj.order()
frame = DataFrame({'b':[4,7,-3,2],'a':[0,1,0,1]})
frame
frame.sort_index(by='b')
frame.sort_index(by=['a','b'])
obj = Series([7,-5,7,4,2,0,4])
obj.rank()
obj.rank(method='first')
obj.rank(ascending=False,method='max')
frame = DataFrame({'b':[4.3,7,-3,2],'a':[0,1,0,1],'c':[-2,5,8,-2.5]})
frame
frame.rank(axis=1)
obj = Series(range(5),index=['a','a','b','b','c'])
obj
obj.index.is_unique
obj['a']
obj['c']
df = DataFrame(np.random.randn(4,3),index=['a','a','b','c'])
df
df.ix['b']
df = DataFrame([[1.4,np.nan],[7.1,-4.5],[np.nan,np.nan],[0.75,-1.3]],index=['a','b','c','d'],columns=['one','two'])
df
df.sum()
df.sum(axis=1)
df.mean(axis=1,skipna=False)
df.idxmax()
df.idxmin()
df.cumsum()
df.describe()
obj = Series(['a','a','b','c']*4)
obj.describe()
import pandas.io.data as web
all_data = {}
for ticker in ['AAPL','IBM','MSFT','GOOG']:
all_data['ticker'] = web.get_data_yahoo(ticker,'1/1/2000','1/1/2010')
price = DataFrame({tic:data['Adj Close'] for tic,data in all_data.iteritems()})
volume = DataFrame({tic:data['Volume'] for tic,data in all_data.iteritems()})
returns = price.pct_change()
returns.tail()
obj = Series([1,2,3,4,5,63,2,11,34])
uniques = obj.unique()
uniques
obj.value_counts()
pd.value_counts(obj.values,sort=False)
mask = obj.isin([1])
mask
obj[mask]
data = DataFrame({'Qu1':[1,3,4,3,4],'Qu2':[2,3,1,2,3],'Qu3':[1,5,2,4,4]})
data
result = data.apply(pd.value_counts).fillna(0)
result
string_data = Series(['aaaaaaa','cccccccc',np.nan,'ddadsds'])
string_data.isnull()
string_data[0] = None
string_data.isnull()
from numpy import nan as NA
data = Series([1,NA,3.5,NA,7])
data.dropna()
data[data.notnull()]
data = DataFrame([[1.,6.5,3.],[1.,NA,NA],[NA,NA,NA],[NA,6.5,3.]])
cleaned = data.dropna()
data
cleaned
data.dropna(how='all')
data[4] = NA
data
data.dropna(axis=1,how='all')
df = DataFrame(np.random.randn(7,3))
df.ix[:4,1] = NA;df.ix[:2,2] = NA
df
df.dropna(thresh=3)
df.fillna(0)
df.fillna({1:0.5,3:-1})
data = Series(np.random.randn(10),index=[['a','a','a','b','b','b','c','c','d','d'],[1,2,3,1,2,3,1,2,2,3]])
data
data.index
data['b']
data[:,2]
data.unstack()
data.unstack().stack()
frame = DataFrame(np.arange(12).reshape((4,3)),index=[['a','a','b','b'],[1,2,1,2]],columns=[['Ohio','Ohio','Colorado'],['Green','Red','Green']])
frame
frame.index.names = ['key1','key2']
frame.columns.names = ['state','color']
frame
frame['Ohio']
frame.swaplevel('key1','key2')
frame.sortlevel(1)
frame.swaplevel(0,1).sortlevel(0)
frame.sum(level='key2')
frame.sum(level='color',axis=1)
开放原子开发者工作坊旨在鼓励更多人参与开源活动,与志同道合的开发者们相互交流开发经验、分享开发心得、获取前沿技术趋势。工作坊有多种形式的开发者活动,如meetup、训练营等,主打技术交流,干货满满,真诚地邀请各位开发者共同参与!
更多推荐
已为社区贡献2条内容
所有评论(0)