Upload
mariano-vicario
View
220
Download
0
Embed Size (px)
Citation preview
8/6/2019 NYC Alt 2010 Nosql
1/52
8/6/2019 NYC Alt 2010 Nosql
2/52
Agenda
What is NoSQL?
NoSQL Databases
Introducing MongoDB .NET Drivers for MongoDB
Introducing NoRM
Case Study: RateMySnippet.com
Questions?
8/6/2019 NYC Alt 2010 Nosql
3/52
NoSQL
8/6/2019 NYC Alt 2010 Nosql
4/52
What is NoSQL?What is NoSQL?
Coined in 1998 by Carlos Strozzi to describe aCoined in 1998 by Carlos Strozzi to describe adatabase that did not expose a SQL interfacedatabase that did not expose a SQL interface
In 2008, Eric Evans reintroduced the term toIn 2008, Eric Evans reintroduced the term todescribe the growing non-RDBMS movementdescribe the growing non-RDBMS movement
Broadly refers to a set of data stores that do notBroadly refers to a set of data stores that do notuse SQL or a relational model to store datause SQL or a relational model to store data
Popularized by large web sites such as Google,Popularized by large web sites such as Google,Facebook and DiggFacebook and Digg
8/6/2019 NYC Alt 2010 Nosql
5/52
NoSQL Databases
Wide Column,Wide Column,Key/ValueKey/Value
and Document Store...and Document Store...Oh My!Oh My!
8/6/2019 NYC Alt 2010 Nosql
6/52
NoSQL DatabasesNoSQL Databases
NoSQL databases come in a variety of flavorsNoSQL databases come in a variety of flavors
XML (myXMLDB, Tamino, Sedna)XML (myXMLDB, Tamino, Sedna)
Wide Column (Cassandra, Hbase, Big Table)Wide Column (Cassandra, Hbase, Big Table) Key/Value (Redis, Memcached with BerkleyDB)Key/Value (Redis, Memcached with BerkleyDB)
Object (db4o, JADE)Object (db4o, JADE)
Graph (neo4j, InfoGrid)Graph (neo4j, InfoGrid) Document store (CouchDB, MongoDB)Document store (CouchDB, MongoDB)
8/6/2019 NYC Alt 2010 Nosql
7/52
Why NoSQL?
67
29
4
Sources of Greenhouse Gas Emissions*
*ZEPA Estimate
Livestock/Agriculture
Automobiles
Relational Databases
8/6/2019 NYC Alt 2010 Nosql
8/52
RDBMSs and the Environment
RDBMS Administrators are highly paid
Highly paid individuals often buy larger than
average homes or cars Larger than average homes and cars require
more energy than smaller home and cars
Therefore RDMBSs contribute to globalwarming more than NoSQL databases whichtypically do not require the addition of a DBA
8/6/2019 NYC Alt 2010 Nosql
9/52
RDBMSs and the Environment
RDBMSs often require high end servers andthat are taxing on disks
High end servers consume more electricity thanmid-range servers
Taxed disks fail more often than untaxed disks
Therefore RDBMSs require more energy andproduce more waste (lots of hard drives inlandfills) than NoSQL DBs, which run on mid-range servers.
8/6/2019 NYC Alt 2010 Nosql
10/52
Why NoSQL?
55
30
15
Leading Causes of Illnesses*
*ZIH Estimate
Contaminants in Food Supply
Overuse of Antibiotics
Relational Databases
8/6/2019 NYC Alt 2010 Nosql
11/52
NoSQL and Healthcare
The current healthcare crisis requires talentedsoftware engineers to fix the outdated or non-existent IT systems of the hospital system
Talented software engineers spend a great dealof time mapping objects to tables in RDBMSs
Talented software engineers are unable to fixhealthcare because they are mapping objectsto tables
Therefore RDBMSs are causing illnessnes
8/6/2019 NYC Alt 2010 Nosql
12/52
Dropping ACID
8/6/2019 NYC Alt 2010 Nosql
13/52
Dropping ACID
CAP Theorem and Eventual Consistency
A distributed computer system cannot provideConsistency, Availability and PartitionTolerance at the same time
Immediate consistency is expensive
Some systems can tolerate dirty reads
Removing relational storage structure helps toreduce transactional requirements
Inserts/Updates written to fewer places
8/6/2019 NYC Alt 2010 Nosql
14/52
Dropping ACIDDropping ACID
ACID support is not required for blogs, statusACID support is not required for blogs, statusupdates, product listings, etc.updates, product listings, etc.
Would a 2 second delay in finding out who theWould a 2 second delay in finding out who the
mayor of Turnpike Spirits is matter?mayor of Turnpike Spirits is matter? Or that your friend is embarrassed to admit thatOr that your friend is embarrassed to admit that
she loves Jersey Shore?she loves Jersey Shore?
Modern relational databases simply do notModern relational databases simply do notscale to Internet proportionsscale to Internet proportions
Sites like Digg, Facebook and Ebay have dataSites like Digg, Facebook and Ebay have datasets 10s or 100s of TB or even PB largesets 10s or 100s of TB or even PB large
8/6/2019 NYC Alt 2010 Nosql
15/52
NoSchema DatabasesNoSchema Databases
NoSQL databases generally do not impose aNoSQL databases generally do not impose aformal schemaformal schema
Optimized storage for unstructured dataOptimized storage for unstructured data
8/6/2019 NYC Alt 2010 Nosql
16/52
NoSchema Documents
Related data is stored in a single document
The ubiquitous Blog example
Each post is a document
Post might have reference to Blog or Userdocument or redundantly store blog/user data
East post document has a nested collection oftags and comments
Application layer must be smarter aboutreferential integrity
8/6/2019 NYC Alt 2010 Nosql
17/52
NoSchema Documents
var post = { Author : { Name : " J ohn Zablock i " , _i d : " 497ce96f. . . " }, Title : " On Running NerdDinner on MongoDB wi th NoRM" , Tags : [" mvc" , " mongodb" , " norm" ] , CreatedDate : new Date(' 6-14-2010' ), Text : " At the Hartford Code Camp. . . "}
8/6/2019 NYC Alt 2010 Nosql
18/52
NoSchema Documents
J ohn Zabl ock i 497ce96f395f2f052a494fd4 On Runni ng NerdDi nner on MongoDB - Part 1 mvc mongodb norm 6-14-2010
At the Hartford Code Camp. . .
8/6/2019 NYC Alt 2010 Nosql
19/52
Something Completely Different
8/6/2019 NYC Alt 2010 Nosql
20/52
Introducing MongoDBIntroducing MongoDB
Open source, document-oriented databaseOpen source, document-oriented database
10gen corporate entity behind development10gen corporate entity behind development
10gen supports official drivers for many10gen supports official drivers for manyplatforms,platforms, but not .NETbut not .NET!!
8/6/2019 NYC Alt 2010 Nosql
21/52
MongoDB The BasicsMongoDB The Basics
Schema-less documents stored in collectionsSchema-less documents stored in collections
Documents are stored as BSON (Binary JSON)Documents are stored as BSON (Binary JSON)
JavaScript used to query and manipulateJavaScript used to query and manipulatedocuments and collectionsdocuments and collections
Each document in a collection has a uniqueEach document in a collection has a unique
BSON ObjectId field named _idBSON ObjectId field named _id Collections belong to a databaseCollections belong to a database
8/6/2019 NYC Alt 2010 Nosql
22/52
MongoDB Server FeaturesMongoDB Server Features
IndexesIndexes
Secondary, compound and uniqueSecondary, compound and unique
ReplicationReplication Master/slave and replica setsMaster/slave and replica sets
Auto-sharding (1.6)Auto-sharding (1.6)
MapReduceMapReduce GridFSGridFS
Large file storageLarge file storage
8/6/2019 NYC Alt 2010 Nosql
23/52
MongoDB Hidden Features
MongoDB will help you:
Make more friends
Be more confident in social situations Be more attractive to members of the opposite
(or same) sex
8/6/2019 NYC Alt 2010 Nosql
24/52
MongoDB on Windows
8/6/2019 NYC Alt 2010 Nosql
25/52
Installing MongoDB on WindowsInstalling MongoDB on Windows
Download the binaries from mongodb.orgDownload the binaries from mongodb.org
Extract to Program Files directory (or wherever)Extract to Program Files directory (or wherever)
Create a directory c:\data\dbCreate a directory c:\data\db Run mongod.exe from the command line with theRun mongod.exe from the command line with the
--install switch--install switch
SeeSee http://bit.ly/aed1RWhttp://bit.ly/aed1RW for some gotchasfor some gotchas
To run the daemon without installing, simply runTo run the daemon without installing, simply runmongod.exe without argumentsmongod.exe without arguments
Run mongo.exe to verify the daemon is runningRun mongo.exe to verify the daemon is running
http://bit.ly/aed1RWhttp://bit.ly/aed1RWhttp://bit.ly/aed1RW8/6/2019 NYC Alt 2010 Nosql
26/52
MongoDB - ShellMongoDB - Shell
The MongoDB interactive JavaScript shellThe MongoDB interactive JavaScript shell(mongo.exe) is a command line utility for(mongo.exe) is a command line utility forworking with MongoDB serversworking with MongoDB servers
Allows for CRUD operations on collectionsAllows for CRUD operations on collections
May be used for basic administrationMay be used for basic administration
Creating indexesCreating indexes
Cloning databasesCloning databases
Also useful as a test-bed while building appsAlso useful as a test-bed while building apps
8/6/2019 NYC Alt 2010 Nosql
27/52
MongoDB - Shell
/*Connect to a server: port/database (defaults are localhost: 27017/test ): */mongo. exe localhost: 27017/AltNetGroup//Switch database:use CodeCamp//View collections in a database:show collections
//create an i ndex on Name fi el ddb. Artists. ensureIndex({ Name : 1 });//copy one database to anotherdb. copyDatabase( " CodeCamp" , " AltNetGroup" )
8/6/2019 NYC Alt 2010 Nosql
28/52
MongoDB Shell CRUDMongoDB Shell CRUD
//Insert an i tem i nto a col lectiondb. Artists. insert({ Name : The Shins });//Fi nd an i tem i n a col lecti on:db. Artists. findOne({ Name: Radiohead});//Find i tems i n a col lection:db. Artists. find({ Name : /The/i});//Count i tems i n a col lectiondb. Artists. count();
8/6/2019 NYC Alt 2010 Nosql
29/52
MongoDB Shell CRUD
//Update an i tem i n a col lectiondb. Artists. update({ Name : J ack J ohnson }, $set : { Name : Jac k Johnson } });/*Update i tems i n a col lectionWithout $set i n thi s exampl e, whol edocument would be replaced! */db. Artists. update({ Name : { $ne : null } }, { $set : { Category : Rock } }, false, true);/*$ denotes special operators and operations$push, $pop, $pul l, etc. */
8/6/2019 NYC Alt 2010 Nosql
30/52
MongoDB MapReduceMongoDB MapReduce
MapReduce is used for batch manipulation andaggregation of data
A Map function transforms a collection into aseries of key/value pairs
A Reduce function operates on that key/valuepair series to produce output based on those
input values An optional Finalize function is called after the
reduce function is invoked
8/6/2019 NYC Alt 2010 Nosql
31/52
MongoDB Shell MapReduce
//Create some datadb. Posts. insert({
Name : " On Instal li ng MongoDB as a Servi ce On Wi ndows" ,Tags : [" mongodb" ]});
db. Posts. insert({ Name : " On Running NerdDinner on MongoDB wi th NoRM" ,
Tags : [" mongodb" , " norm" , " mvc" ]});
db. Posts. insert({ Name : " On A Si mpl e IronPython Route Mapper for ASP. NET MVC" ,Tags : [" mvc" , " ironpthon" ]
});
8/6/2019 NYC Alt 2010 Nosql
32/52
MongoDB Shell MapReduceMongoDB Shell MapReduce
/*Create the map functionThi s functi on creates a vi ew that look s li k e{ mvc, [1, 1] },{ norm, [1] }*/var map = function() {
if (! this. Tags) { return; } for (var index in this. Tags) {
emit(this. Tags[index] , 1); } } ;
8/6/2019 NYC Alt 2010 Nosql
33/52
MongoDB Shell MapReduceMongoDB Shell MapReduce
/*Create the reduce functionconceptual ly, reduce gets cal led l i k e:reduce( " mvc" , [1, 1] );reduce( " norm" , [1]/*var reduce = function( ey, vals) { var count = 0; for (var index in vals) { count += vals[index] ; } return count; } ;
8/6/2019 NYC Alt 2010 Nosql
34/52
MongoDB Shell MapReduceMongoDB Shell MapReduce
/*Run the mapreduce command on the Posts col lectionusing the map and reduce functions defined abovestore the resul ts i n a col lecti on named Tags*/var result = db. runCommand( { mapreduce : " Posts " , map : map, reduce : reduce, out : " Tags " });db. Tags. find()
8/6/2019 NYC Alt 2010 Nosql
35/52
.NET and MongoDB
8/6/2019 NYC Alt 2010 Nosql
36/52
.NET and MongoDB
No officially supported 10gen driver
mongo-csharp
Actively being developed and supported Supports typed collections
Simple-mongodb
Json-centric Actively developed, but less so than other
drivers like mongo-csharp and NoRM
8/6/2019 NYC Alt 2010 Nosql
37/52
NoRM
8/6/2019 NYC Alt 2010 Nosql
38/52
NoRMNoRM
OK, it is Pronounced No R M!OK, it is Pronounced No R M!
Actively developed by Andrew Theken andActively developed by Andrew Theken and
James Avery, among othersJames Avery, among others Active community with reliable supportActive community with reliable support
I received help even as I prepared this slide!I received help even as I prepared this slide!
Support for typed and untyped collections,Support for typed and untyped collections,MapReduce, Property Mapping, LINQ,MapReduce, Property Mapping, LINQ,Expando, nearly all CRUD operationsExpando, nearly all CRUD operations
8/6/2019 NYC Alt 2010 Nosql
39/52
NoRM The BasicsNoRM The Basics
//Connections managed with IDisposable pattern//Connect to test database on l ocalhostusing (Mongo mongo = Mongo. Create( " mongodb: //localhost/test" )){
Console. WriteLine(mongo. Database. DatabaseName);}//Mongo instance has MongoDatabase property/*MongoDatabase has GetCollection methods for accessing MongoCollection instances*///CRUD operations performed on collections
N RM CRUD
8/6/2019 NYC Alt 2010 Nosql
40/52
NoRM - CRUDNoRM - CRUD
//Class properties map to document structure public class Artist { //ObjectId property mapped to _id field i n document public ObjectId Id { get; set; } public string Name { get; set; } //IList property wi l l be mapped to J avaScript Array private IList _albums = new List(0); public IList Albums { get { return _albums ; } set { _albums = value; } } }
CN RM CRUD
8/6/2019 NYC Alt 2010 Nosql
41/52
NoRM - CRUDNoRM - CRUD
var artist = new Artist() { Name = " BadLands" };//Inserting a document i nto a typed col lection mongo. Database. GetCollection( " Artists" ). Insert(artist);//Updating (replacing) a document i n a typed col lectionartist. Name = " Badlands" ; mongo. Database. GetCollection( " Artists" ). Save(artist);//Updating a nested collectionmongo. Database. GetCollection( " Artists" ). UpdateOne(new { Name = " Badlands" },
new { Albums = M. Push( " Voodoo Highway" ) } );
N RM CRUDN RM CRUD
8/6/2019 NYC Alt 2010 Nosql
42/52
NoRM - CRUDNoRM - CRUD
//Find al l documents i n a typed col lectionvar artists = mongo. GetCollection( " Artists" ). Find();Console. WriteLine(artists. FirstOrDefault(). Name);//Query with a document specvar artist = mongo. GetCollection( " Artists" ). FindOne(new { Name = " Badlands" });Console. WriteLine(artist. Albums. Count);//Count the documents i n a col lectionlong count = mongo. GetCollection( " Artists" ). Count();Console. WriteLine(count);
N RM M R dN RM M R d
8/6/2019 NYC Alt 2010 Nosql
43/52
NoRM - MapReduceNoRM - MapReduce
//Add a Tags col lection added to Artist cl assprivate IList _tags; public IList Tags {
get { return _tags; }set { _tags = value; }
}//Add some tags use Set not PushAll, si nce Tags was null mongo. Database. GetCollection( " Artists" ). UpdateOne(
new { Name = " Badlands" },new { Tags = M. Set(new List(){ " Hard Rock " , " 80s " }) });
N RM M R dN RM M R d
8/6/2019 NYC Alt 2010 Nosql
44/52
NoRM - MapReduceNoRM - MapReduce
//Create map and reduce functonsstring map = @" function() {if (! this. Tags ) { return; } for (index in this. Tags) {
emit(this. Tags[index] , 1); } } " ;
string reduce = @" function(previous, current) {var count = 0;
for (index in current) { count += current[index];
} return count; } " ;
N RM M R dN RM M R d
8/6/2019 NYC Alt 2010 Nosql
45/52
NoRM - MapReduceNoRM - MapReduce
//MapReduce class is responsible for calling mapreduce commandMapReduce mr = mongo. Database. CreateMapReduce(); //Represents the document passed to the//db. runCommand in the shell exampleMapReduceOptions options = new MapReduceOptions( " Artists" ){
Map = map,Reduce = reduce,OutputCollectionName = " Tags "
}; MapReduceResponse response = mr. Execute(options);var collection = mongo. Database. GetCollection( " Tags " );Console. WriteLine(collection. Count());
N RM LINQNoRM LINQ
8/6/2019 NYC Alt 2010 Nosql
46/52
NoRM - LINQNoRM - LINQ
//LINQ provider exposed via AsQueryable method of MongoCollectionvar artists = mongo. Database. GetCollection( " Artists" ). AsQueryable(); //Find i tems i n typed col lectionvar artistsStartingWithB =
from a in mongo. Database. GetCollection(). AsQueryable()where a. Name. StartsWith( B )select a;
Console. WriteLine( " Fi rst arti st starti ng wi th B: " +artistsStartingWithB. First(). Name);
NoRM LINQNoRM LINQ
8/6/2019 NYC Alt 2010 Nosql
47/52
NoRM - LINQNoRM - LINQ
//Find artists without pul ling back nested col lections//Note use of Regex name searchvar artistsWithoutAlbums =
from a in mongo. Database. GetCollection(). AsQueryable()where Regex. IsMatch(a. Name, " land" , RegexOptions. IgnoreCase)select new { Name = a. Name };Console. WriteLine(artistsWithoutAlbums . First(). Name);
//Fi nd arti sts wi th a gi ven tagvar artistsWithHardRock Tag =mongo. Database. GetCollection( " Artists" ). AsQueryable(). Where(a => a. Tags. Any(s => s == " Hard Rock " ));Console. WriteLine(artistsWithHardRock Tag. First(). Name);
Case Study: RateMySnippet com
8/6/2019 NYC Alt 2010 Nosql
48/52
Case Study: RateMySnippet.com
Final Thoughts
8/6/2019 NYC Alt 2010 Nosql
49/52
Final Thoughts
Eat food. Not too much. Mostly plants.- Michael Pollan
Final Thoughts
8/6/2019 NYC Alt 2010 Nosql
50/52
Final Thoughts
Write Code. Not too much. Mostly C#.- John Zablocki
Final Thoughts The Stack
8/6/2019 NYC Alt 2010 Nosql
51/52
Final Thoughts The Stack
ASP.NET MVC
MVCContrib
MongoDB
NoRM SQL Server
NHibernate FluentNHibernate
Spring.NET Recoil
AutoMapper
Q & A
8/6/2019 NYC Alt 2010 Nosql
52/52
Q & A
http://dllHell.net - my blog
http://www.CodeVoyeur.com - my code
http://www.linkedin.com/in/johnzablocki http://twitter.com/codevoyeur
http://mongodb.org - Official MongoDB site
http://groups.google.com/group/norm-mongodb http://www.summerofnorm.com - Coming Soon!
http://dllhell.net/http://www.codevoyeur.com/http://www.linkedin.com/in/johnzablockihttp://twitter.com/codevoyeurhttp://mongodb.org/http://groups.google.com/group/norm-mongodbhttp://www.summerofnorm.com/http://www.summerofnorm.com/http://groups.google.com/group/norm-mongodbhttp://mongodb.org/http://twitter.com/codevoyeurhttp://www.linkedin.com/in/johnzablockihttp://www.codevoyeur.com/http://dllhell.net/