Spring Data y Mongo DB en un proyecto Real

David Gómez@dgomezg

Spring Data y MongoDBen un proyecto real.

sábado 18 de febrero de 12

David Gómez

dgomez@extrema-sistemas.comSw Architect & Trainer

@dgomezg

Spring CORE

Rich Web Applications

with Spring

Enterprise Integration

#springio

#extrema

Hablaremos de

Un caso real

¿Por qué NoSQL? ¿Por qué MongoDB?

MongoDB ‘a pelo’

Spring Data al rescate

¡Cuidado!

Un caso real. Control de tráfico marítimo

Emisor AIX

3-5 min

Emisor AIX

3-5 min

Emisor AIX

3-5 min

Receptor GPS

Goniometro

Estación Meteo,

Cámaras,

Radar (3 s)

El problema: La escalabilidad

Por cada barco:

Recepción y procesamiento de señales (nº)

Transacciones,

Cálculo de alarmas

Gestión de Históricos

El problema: La escalabilidadSeñales

(1 AIS x 3 min + 1 radar x 3 seg) x 100 barcos

Alarmas

Persistencia de Líneas de Varada

(10.000+ ptos x 100+ líneas)

40 Alarmas

Zonas, Elementos Navegación

Persistencia: Total: 87 tablas + históricos(x2)

¿Por qué NoSQL?

Teorema CAP (Eric Brewer)

Source: Nathan Hurst’s Blog

¿Por qué NoSQL?

CConsistency

¿Por qué NoSQL?

C AConsistency Availability

¿Por qué NoSQL?

Consistency Availability

Partition Tolerance

¿Por qué NoSQL?

Partition Tolerance

¿Por qué NoSQL?

Partition Tolerance

Sólo 2

¿Por qué NoSQL?

Partition Tolerance

Sólo 2

Oracle, MySQL,

¿Por qué NoSQL?

Partition Tolerance

Sólo 2

Oracle, MySQL,

MongoDB, DataStore,

HyperTable, HBase

Redis, Mem

cacheDB

¿Por qué NoSQL?

Partition Tolerance

Sólo 2

Oracle, MySQL,

MongoDB, DataStore,

HyperTable, HBase

Redis, Mem

cacheDBDyn

amo, V

Simple

DB, Cou

RDBMSOrientadas a documento

orientadas a columnaKey-Value

Pero... ¿NoSQL?

(cc) Photo by mwin - http://www.flickr.com/photos/wingler/3429634150

Pero... ¿NoSQL?

NoSQL !== no SQL

Not only SQL

NoSQL: ACID vs BASE

Atomicity,

Consistency,

Isolation,

Durability

(cc) Photo by zhouxuan12345678 - http://www.flickr.com/photos/53921113@N02/5645102295/

Basically Available,

Soft state,

Eventual consistency

orientadas a Columna orientadas a Documento

orientadas a GrafosKey-value

Tipos.

¿Por qué MongoDB?

Orientada a documento

Almacenados en Colecciones

(bajo una clave)

Los documentos pueden ser heterogéneos

Formatos textuales: JSON & BSON

(cc) Photo by Ampersand Duck - http://www.flickr.com/photos/ampersandduck/4941185476

MongoDB

Escrita en C++

Orientada a documento

Formato JSON (o BSON)

Un poco SQL (queries, índices, Referencias externas)

Particionado horizontal (sharding)

Consultas Javascript

(cc) Photo by Ampersand Duck - http://www.flickr.com/photos/ampersandduck/4941185476

Orientadas a Documento

Documentos formateados en JSON

Orientadas a Documento

Documentos formateados en JSON

{ "_id" : "224000999", "_class" : "com.vts.model.Vessel", "flag" : "ALBANIA", "name" : "Sample NOT_AVAILABLE Vessel 224000999", "callsign" : "SV224000999", "toBow" : 25, "toStern" : 5, "toPort" : 5, "toStarboard" : 5, "comments" : "Sample vessel created automatically for test purposes"}

El API

Consola y API JavaScript:

El API

Consola y API JavaScript:beleriand:bin dgomez$ ./mongoMongoDB shell version: 1.8.1connecting to: test>

El API

Consola y API JavaScript:beleriand:bin dgomez$ ./mongoMongoDB shell version: 1.8.1connecting to: test> > use vtsswitched to db vts>

El API

Consola y API JavaScript:beleriand:bin dgomez$ ./mongoMongoDB shell version: 1.8.1connecting to: test> > use vtsswitched to db vts>> show collectionsEventWeatherDatasystem.indexesvessel>

El API

Consola y API JavaScript:beleriand:bin dgomez$ ./mongoMongoDB shell version: 1.8.1connecting to: test> > use vtsswitched to db vts>> show collectionsEventWeatherDatasystem.indexesvessel>> db.Event.find(){ "_id" : ObjectId("4e0b5b211446446f6be3bb1a"), "_class" : "com.vts.model.events.SystemEvent", "timestamp" : ISODate("2011-06-29T17:04:33.039Z") }{ "_id" : ObjectId("4e0b5b3d144676f49946443f"), "_class" : "com.vts.model.events.SystemEvent", "timestamp" : ISODate("2011-06-29T17:05:01.394Z") }>

El API

Consola y API JavaScript:

El API

Consola y API JavaScript:> show collectionsEventWeatherDatasystem.indexesvessel>

El API

Consola y API JavaScript:> show collectionsEventWeatherDatasystem.indexesvessel> > db.alarms.save({"type" : "speed", "severity" : "warn", "vesselSSID" : "224000999"})>

El API

Consola y API JavaScript:> show collectionsEventWeatherDatasystem.indexesvessel> > db.alarms.save({"type" : "speed", "severity" : "warn", "vesselSSID" : "224000999"})>> show collectionsEventWeatherDataalarmssystem.indexesvessel>

El API

Consola y API JavaScript:> show collectionsEventWeatherDatasystem.indexesvessel> > db.alarms.save({"type" : "speed", "severity" : "warn", "vesselSSID" : "224000999"})>> show collectionsEventWeatherDataalarmssystem.indexesvessel>> db.alarms.find(){ "_id" : ObjectId("4f3acca3ed4cc8078fd1a4be"), "type" : "speed", "severity" : "warn", "vesselSSID" : "224000999" }>

Documentos

Identificados por un “_id” (generado o asignado)

Textual (JSON)

Binario (BSON)*

Documentos grandes divididos en chunks

Programación de clientes

Multitud de lenguajes (drivers)

Transformación a JSON realizada por el driver

{ "_id" : "224000999", "flag" : "ALBANIA", "name" : "Sample NOT_AVAILABLE Vessel 224000999", "callsign" : "SV224000999", "toBow" : 25, "toStern" : 5, "toPort" : 5, "toStarboard" : 5, "comments" : "Sample vessel created automatically for test purposes" }

sample.py

{ "_id" => "224000999", "flag" => "ALBANIA", "name" => "Sample NOT_AVAILABLE Vessel 224000999", "callsign" => "SV224000999", "toBow" => 25, "toStern" => 5, "toPort" => 5, "toStarboard" => 5, "comments" => "Sample vessel created automatically for test purposes" }

sample.py

array( "_id" => "224000999", "flag" => "ALBANIA", "name" => "Sample NOT_AVAILABLE Vessel 224000999", "callsign" => "SV224000999", "toBow" => 25, "toStern" => 5, "toPort" => 5, "toStarboard" => 5, "comments" => "Sample vessel created automatically for test purposes"}

sample.php

sample.py

array( "_id" => "224000999", "flag" => "ALBANIA", "name" => "Sample NOT_AVAILABLE Vessel 224000999", "callsign" => "SV224000999", "toBow" => 25, "toStern" => 5, "toPort" => 5, "toStarboard" => 5, "comments" => "Sample vessel created automatically for test purposes"}

sample.php

BasicDBObject doc = new BasicDBObject(); doc.put("flag", vessel.getFlag()); doc.put("name", vessel.getName()); doc.put("callsign", vessel.getCalSign()); doc.put("toBow", vessel.getBow()); doc.put("toPort", vessel.getStern()); doc.put("toStern", vessel.getPort()); doc.put("toStartboard", vessel.getStarboard()); doc.put("comments", vessel.getComments());

sample.java

Cliente Java con MongoDB

‘a pelo’

1. Obtener el Driver

Importar el driver

1. Obtener el Driver

Importar el driver

<dependency><groupId>org.mongodb</groupId><artifactId>mongo-java-driver</artifactId><version>2.7.0-rc1</version>

</dependency>

2. Conexión

Obtener la referencia a la BD y la conexión

2. Conexión

Obtener la referencia a la BD y la conexión

Mongo m = new Mongo( "127.0.0.1" , 27017 ); DB db = m.getDB( "playground" );

3. Inserciones

Crear e insertar un objeto

3. Inserciones

Crear e insertar un objeto

BasicDBObject doc = new BasicDBObject(); doc.put("flag", vessel.getFlag()); doc.put("name", vessel.getName()); doc.put("callsign", vessel.getCalSign()); doc.put("toBow", vessel.getBow()); doc.put("toPort", vessel.getStern()); doc.put("toStern", vessel.getPort()); doc.put("toStartboard", vessel.getStarboard()); doc.put("comments", vessel.getComments());

DBCollection coll = db.getCollection("vessel"); coll.insert(doc);

4. Consultas (I)

Obtener todos los documentos de una colección

DBCollection vessels = db.getCollection("vessel");

DBCursor cursor = vessels.find(); List<Vessel> vessels = new ArrayList<>();while (cursor.hasNext()) { DBObject jsonVessel = cursor.next(); Vessel vessel = new Vessel(); vessel.setMmsi(jsonVessel.get(“mmsi”)); vessel.setLength(jsonVessel.get(“length”));

// ...

vessels.add(vessel);}

4. Consultas (I)

DBCursor cursor = vessels.find(); List<Vessel> vessels = new ArrayList<>();while (cursor.hasNext()) { DBObject jsonVessel = cursor.next(); Vessel vessel = new Vessel(); vessel.setMmsi(jsonVessel.get(“mmsi”)); vessel.setLength(jsonVessel.get(“length”));

// ...

vessels.add(vessel);}

4. Consultas (I)

public abstract class DBCollection {

public final DBCursor find(){ public final DBCursor find( DBObject ref ) public final DBCursor find( DBObject ref , DBObject keys )

public final DBObject findOne( Object obj ) public final DBObject findOne( Object obj, DBObject fields )

public DBObject findAndModify( DBObject query , DBObject update )

4. Consultas (II)

Consultas “SQL-like”

4. Consultas (II)

Consultas “SQL-like” En RDBMS: SELECT * FROM VESSEL WHERE FLAG = “ALBANIA”;

Las consultas se hacen también en JSON

4. Consultas (II)

{ “flag” : “ALBANIA” }

4. Consultas (II)

beleriand:bin dgomez$ ./mongoMongoDB shell version: 1.8.1connecting to: test> use vtsswitched to db vts> db.vessel.find( { "flag" : "ALBANIA" });{ "_id" : "224312999", "_class" : "com.vts.model.Vessel", "flag" : "ALBANIA", "name" : "Sample TOWING Vessel 224312999", "callsign" : "SV224312999", “length”: 30, “beam” : “10”, "toBow" : 25, "toStern" : 5, "toPort" : 5, "toStarboard" : 5, "comments" : "Sample vessel created automatically for test purposes" }>

4. Consultas (II)

beleriand:bin dgomez$ ./mongoMongoDB shell version: 1.8.1connecting to: test> use vtsswitched to db vts> db.vessel.find( { "flag" : "ALBANIA" });{ "_id" : "224312999", "_class" : "com.vts.model.Vessel", "flag" : "ALBANIA", "name" : "Sample TOWING Vessel 224312999", "callsign" : "SV224312999", “length”: 30, “beam” : “10”, "toBow" : 25, "toStern" : 5, "toPort" : 5, "toStarboard" : 5, "comments" : "Sample vessel created automatically for test purposes" }>

DBObject query = new BasicDBObject();query.put("flag", "ALBANIA"); DBCursor albanianShips = vessels.find(query);while (albanianShips.hasNext()) { // ...}

Operadores de consulta

Otros operadores

Operadores de consulta

Otros operadores

$all$and$exists$gt$in$lt$mod$ne$nin$nor$or$size$type

Consultas complejasConcatenación de consultas

RDBMS:

SELECT * FROM VESSELS WHERE length < 90 AND length > 30AND (flag = “ALBANIA” or flag = ”SAN MARINO”);RDBMS:

SELECT * FROM VESSELS WHERE length < 90 AND length > 30AND (flag = “ALBANIA” or flag = ”SAN MARINO”);

{ “length” : { $lt : 90, $gt : 30}, $or : [ { “flag” : “ALBANIA”}, { “flag” : “SAN MARINO”} ]}

RDBMS:

Mongo:

SELECT * FROM VESSELS WHERE length < 90 AND length > 30AND (flag = “ALBANIA” or flag = ”SAN MARINO”);

{ “length” : { $lt : 90, $gt : 30}, $or : [ { “flag” : “ALBANIA”}, { “flag” : “SAN MARINO”} ]}

DBObject query = new BasicDBObject();DBObject condition = new BasicDBObject("$lt",90);condition.put("$gt",30);query.put("length", condition);

DBObject [] flags = {new BasicDBObject("flag", "ALBANIA"), new BasicDBObject("flag", "SAN MARINO")

};query.put("$or", flags);

talks.find(query);

RDBMS:

Mongo:

Driver:

DBObjects desde JSON

com.mongo.util.JSON

DBObjects desde JSON

Object query = JSON.parse("{ 'length' : { $lt : 90, $gt : 30}, " + " '$or' : [ { 'flag' : 'ALBANIA}, " + " { 'flag' : 'SAN MARINO'} ] }");

DBCursor cursor = db.find((DBObject) query);

com.mongo.util.JSON

Además...

Gestión de excepciones

Gestión recursos

DB, Collección, Cursores, DBObjects...

Transformaciones Objeto JSON

Spring Data MongoDB

Spring DataObjetivo:

Proporcionar un mecanismo homogéneo y completo para el acceso a BD NoSQL

Multiples proyectos:

Soportados:

MongoDB, Redis, Neo4j, Hadoop, GemFire, RiaK

En proceso:

CouchDB, Hbase, Cassandra

Spring Data y MongoDB

XML namespace para configurar driver Mongo

MongoTemplate

Conversión de excepciones automática

Conversión Configurable

JMX monitoring

Importando Spring Data

<repository> <id>spring-release</id> <name>Spring Maven Release Repository</name> <url>http://repo.springsource.org/libs-release</url></repository>

<dependency> <groupId>org.springframework.data</groupId> <artifactId>spring-data-mongodb</artifactId> <version>1.0.1.RELEASE</version></dependency>

Namespace

<?xml version="1.0" encoding="UTF-‐8"?><beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-‐instance" xmlns:mongo="http://www.springframework.org/schema/data/mongo" xmlns:context="http://www.springframework.org/schema/context" xsi:schemaLocation="http://www.springframework.org/schema/beans

http://www.springframework.org/schema/beans/spring-‐beans.xsd http://www.springframework.org/schema/data/mongo http://www.springframework.org/schema/data/mongo/spring-‐mongo-‐1.0.xsd

http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-‐context.xsd">

</beans>

Namespace

</beans>

<mongo:db-‐factory dbname="vts"/>

Namespace

</beans>

<mongo:db-‐factory dbname="vts"/>

MongoTemplate

Identifica la colección

Convierte query a JSON

Convierte respuesta (“_class”)

Gestiona las conexiones y cursores

Convierte las excepciones

MongoTemplate - Consultas

public class MongoTemplate implements MongoOperations, ApplicationContextAware {

public <T> T findOne(Query query, Class<T> entityClass)

public <T> List<T> find(Query query, Class<T> entityClass)

public <T> T findById(Object id, Class<T> entityClass)

public <T> List<T> findAll(Class<T> entityClass)

public <T> T findOne(Query query, Class<T> entityClass)

public <T> List<T> find(Query query, Class<T> entityClass)

public <T> T findById(Object id, Class<T> entityClass)

public <T> List<T> findAll(Class<T> entityClass)

} public <T> T findOne(Query query, Class<T> entityClass, String collectionName)

Consultas

SELECT * FROM VESSEL WHERE FLAG = “ALBANIA”; RDBMS:

Consultas

SELECT * FROM VESSEL WHERE FLAG = “ALBANIA”;

RDBMS:

Mongo:

Consultas

DBObject query = new BasicDBObject();query.put("flag", "ALBANIA"); DBCursor albanianShips = vessels.find(query);while (albanianShips.hasNext()) { DBObject result = albanianShips.next(); // Create Vessel object from DBObject}

RDBMS:

Mongo:

Driver:

Consultas

Query query = new Query(Criteria.where("flag").is("ALBANIA");

List<Vessel> vessels = mongoTemplate.find(query, Vessel.class);

Utilizado para

Identificación de la colección

Serialización JSON

RDBMS:

Mongo:

Serialización automática

{ "_id" : "224000999", "_class" : "com.vts.model.Vessel", "flag" : "ALBANIA", "name" : "Sample NOT_AVAILABLE Vessel 224000999", "callsign" : "SV224000999", "toBow" : 25, "toStern" : 5, "toPort" : 5, "toStarboard" : 5, "comments" : "Sample vessel created automatically for test purposes" }

SELECT * FROM VESSELS WHERE length < 90 AND length > 30ORDER BY lastUpdate;

{ “length” : { $lt : 90, $gt : 30}}

DBObject query = new BasicDBObject();DBObject condition = new BasicDBObject("$lt",90);condition.put("$gt",30);query.put("length", condition);

DBObject [] flags = {new BasicDBObject("flag", "ALBANIA"), new BasicDBObject("flag", "SAN MARINO")

};query.put("$or", flags);

vessels.find(query);

{ “length” : { $lt : 90, $gt : 30}}

Criteria criteria = Criteria.where("length");criteria.gte(from);criteria.lte(to);

Query query = new Query(criteria);query.sort().on("lastUpdate", Order.ASCENDING);

List<Vessel> vessels = mongoTemplate.find(query, Vessel.class);

MongoTemplate - inserciones

public void save(Object objectToSave)

public void save(Object objectToSave, String collectionName)

AbstractRepository

public abstract class AbstractMongoRepository<T> {

@Inject protected MongoTemplate mongoTemplate; /** Clase que utiliza Mongo para identificar el nombre de la coleccion */ private Class<T> persistentClass;

protected AbstractService(Class<T> persistentClass) { this.persistentClass = persistentClass; }

public String determineCollection() { return persistentClass.getSimpleName(); }}

AbstractRepository

AbstractRepositorypublic abstract class AbstractMongoRepository<T> {

public void save(T e) { mongoTemplate.save(e, determineCollection()); } public void save(Collection<T> entities) { for (T e : entities) { save(e); } } public void removeById(String id) { Object e = findById(id); if (e != null) { mongoTemplate.remove(e, determineCollection()); } } public T findById(String id) { T t = mongoTemplate.findById(id, persistentClass); if (t == null) { throw new ObjectNotFoundException(persistentClass, id); } return t; }}

VesselRepository

public class VesselRepository extends AbstractMongoRepository<Vessel> {

public VesselRepository() { super(Vessel.class); }

A tener en cuenta

El caso de BitSet

Cuidado con los campos transient

El caso de BitSet

public class BitSet implements Cloneable, java.io.Serializable {

/** The internal field corresponding to the serialField "bits". */ */ private long[] words;

/** The number of words in the logical size of this Bitset */ private transient int wordsInUse = 0;

private void expandTo(int wordIndex) { int wordsRequired = wordIndex+1; if (wordsInUse < wordsRequired) { ensureCapacity(wordsRequired); wordsInUse = wordsRequired; } }}

Cuidado con los campos transient

Conversión Manual

public class BitSetReadConverter implements Converter<DBObject, BitSet> {

@Override public BitSet convert(DBObject source) { BasicDBList words = (BasicDBList)source.get("words"); BitSet bitset = new BitSet(); int index = 0; for (Object word : words) { long value = (Long)word; while (value != 0L) {

if ((value & 1L) != 0) { bitset.set(index); } ++index; value = value >>> 1; } } return bitset; }}

Conversión ManualPorque a veces lo automático no funciona

<?xml version="1.0" encoding="UTF-8"?><beans>

</beans>

Queries con multiples valores

Query query = new Query( Criteria .where("length").gte(20) .and("length").lt(60));

DEBUG MongoTemplate - find using query: { "length" : { "$lt" : 60} }

Query query = new Query( Criteria .where("length").gte(20).lte(60));

DEBUG MongoTemplate - find using query: { "toBow" : { "$gte" : 20 , "$lte" : 60} }

Conclusiones

Reducción del número de tablas (87x2 vs 5)

Optimización en el rendimiento de escrituras

Simplificación de los DAOs con Spring-Data

Otras ventajas:

Documentos en formato de Respuesta REST

interfaz Javascript en la consola

David Gómez@dgomezg

Spring Data y Mongo DB en un proyecto Real

Technology

Mongo db 特性

NoSQL, Mongo DB & Google Maps

Javaでmongo db

07.2-Mongo DB 1

Mongo db &lamp;redis,nosql

Mongo DB y Mongo DB atlas - pandorafms.com

Mongo db with C#

[110730/아꿈사발표자료] mongo db 완벽 가이드 : 7장 '고급기능

Wilver mongo db

Mongo DB 성능최적화 전략

1 tutorial de mongo db

Curso mongo db com php

Minicurso mongo db

50 tips & tricks for mongo db developers

Mongo db 2.x to 3.x

Pentaho＋mongo db勉強会20150416

Mongo db course administration

¿Cómo funciona la replicación en mongo db?

Mongo db 복제(Replication)

Mongo db 2장