23
Big Data: тренды, кейсы и технологии Павловский Евгений Николаевич, к.ф.-м.н., научный сотрудник НГУ директор ООО Исследовательские системы (xpss.ru)

Одна лекция из мира Big Data: тренды, кейсы и технологии

Embed Size (px)

DESCRIPTION

Лекция прочитана мной для госслужащих Новосибирской области 10 октября 2013 года в Центре дополнительного образования НГУ

Citation preview

  • 1. Big Data:, , ..-.., (xpss.ru)

2. ? ? ? ? ? ? ? , () BigData? ? 3. BigData? , , ( , Google Now) 4. Big Data: ? , , , ( .. ) ? EMC Oracle IBM Amazon 10000 / , Cloudera - -- ? , - ? - (, ) - Google FS Hadoop 5. Big Data? Volume 1Gb,1Tb, 1Pb, 1Exb, 1 ZettaByteVariety DB, XML, Logs, Texts (.doc, .xls, .ppt ), Audio, Video Value $5 FaceBook - ( ) $3M Intel Core, 2014 $30M 6. Volume 7. Variety 8. (Value) McKinsey Global Institute (2011) $300. US Private Sector 60% Europe admin savings $149. ROI IT- - 9. : (2011) 34% , (PricewaterhouseCoopers). 37%, 60% $100 . $80 Visa 50 . 500 . $2 . 10. : 2012 During the 1,5 year prior to the Election Day in November 2012 in total over $ 1.5 billion was collected and spent during the Obama campaign. In addition, over 1.000 paid staff worked on the campaign, well over 10.000s volunteers and in total more than 100 data analysis who ran more than 66,000 computer simulations every day. The objective of the campaign set out by Jim Messina was to measure everything. The idea was to demand data on everything that happened during the campaign in order to measure everything and ensure that they were being smart about everything. 11. Data Science & Engineering - $300. $5000 . 2-4 (Python, Ruby) UNIX Google Big Table key-value databases - . . .. 12. Data Scientists? ? .., . - - 13. Data Scientists "Data scientists turn big data into big value, delivering products that delight users, and insight that informs business decisions. Strong analytical skills are a given: above all, a data scientist needs to be able to derive robust conclusions from data. But a data scientist also needs to possess creativity and strong communication skills". Daniel Tunkelang, Principal Data Scientist, LinkedIn "A data scientist is someone who can obtain, scrub, explore, model and interpret data, blending hacking, statistics and machine learning. Data scientists not only are adept at working with data, but appreciate data itself as a first-class product". Hilary Mason, Chief Scientist at bitly 14. Big Data (CAPEX) (value) (OPEX) Data Scientist Data Engineer Manager Hadoop - , Splunk PreCog BigML 15. BUSINESS UNDERSTANDINGDATA UNDERSTANDINGDATA PREPARATION DEPLOYMENTDataEVALUATIONMODELING 16. ? Open Data http://data.mos.ru/ : 194 34 14 http://opengovdata.ru http://hubofdata.ru 5 260 2007 2013 2GIS API , Flamp API (Linked Science) , Real Time Billing 17. http://bit.ly/HBRbigdatahttp://bit.ly/BigDataRoadmaphttp://bit.ly/CRUbigdata 18. ? HBR , ( ) - , ( ) , - , 19. , ..-.. () +79139117907 [email protected] Skype: eunipav