Couchbase 102 - SDK Operations

Technical  Evangelist

twi0er:  @scalabl3email:  jasdeep@couchbase.com

Jasdeep  Jaitla

Couchbase  102:  SDK  OperaFons

SETUP  SDK

Supported SDK'swww.couchbase.com/communi/es

• Each  supported  SDK  page  has  instrucFons  for  setup  

• PHP,  Ruby,  NodeJS  and  Python  clients  are  wrappers  around  libcouchbase  C  library,  so  libcouchbase  must  be  installed  first  

• For  other  community  clients,  click  on  "All  Clients"  on  leS  nav,  scroll  down  the  page  and  you  can  see  clients  for  Go,  Erlang,  Clojure,  TCL,  other  nodejs  and  Perl.

Installing Libcouchbase

• Mac  Tips  before  Libcouchbase  &  SDK  Install  

• Make  sure  you  have  XCode  &  Command  Line  Tools  Installed  

• Install  Homebrew  if  you  don't  have  it  already  

• Do  a  $  brew  update,  $  brew  upgrade  and  $  brew  doctor  to  be  sure  you're  up  to  date

www.couchbase.com/communi/es/c/geBng-­‐started

Installing Libcouchbase

• Mac  Via  Homebrew  

• $  brew  install  libcouchbase  

• PC-­‐Windows    

• Download  appropriate  Zip  from  website  

• Redhat/CentOS  

• wget  the  yum  repositories  

• $  sudo  yum  install  -­‐y  libcouchbase2-­‐libevent  libcouchbase-­‐devel  

• Ubuntu  

• wget  ubuntu  repositories  

• $  sudo  apt-­‐get  install  libcouchbase2-­‐libevent  libcouchbase-­‐dev

www.couchbase.com/communi/es/c/geBng-­‐started

Client Connections

Couchbase Server8091

11210

8092

11211Application Server

Client Connections

Couchbase Server8091

11210

8092

11211Application Server

HTTP

Cluster  Configura/on  and  Cluster  Par//on  Map

Client Connections

Couchbase Server8091

11210

8092

11211Application Server

HTTP

Cluster  Configura/on  and  Cluster  Par//on  Map

MAP

Client Connections

Couchbase Server8091

11210

8092

11211Application Server

HTTP

MAP

Client Connections

Couchbase Server8091

11210

8092

11211Application Server

HTTP

MAP

TCP Binary

Create  Socket  Connec/on  to  Each  Node  in  Cluster

Client Connections

Couchbase Server8091

11210

8092

11211Application Server

HTTP

MAP

TCP Binary

Client Connections

Couchbase Server8091

11210

8092

11211

HTTP

Application Server

HTTP

MAP

View  Querying

TCP Binary

Client Connections

Couchbase Server8091

11210

8092

11211

HTTP

Application Server

HTTP

MAP

TCP Binary

Make a Connectionrequire 'rubygems'!require 'couchbase'! !cb = Couchbase.connect(!

:bucket => "default",!:hostname => "localhost")!!

data = { jsonkey: "value", created_at: Time.now }!!cb.add("mydata", data)!puts cb.get("mydata")

#!/usr/bin/env python!from couchbase import Couchbase!from pprint import pprint!from datetime import datetime!!cb = Couchbase.connect(bucket='default')!!data = { "jsonkey": "value", "created_at": datetime.now() }!!cb.add('mydata', data)!result = cb.get('mydata')!pprint(result.value, indent=4)!

RUBY

PYTHON

Make a Connectionimport com.couchbase.client.CouchbaseClient;!import java.net.URI;!import java.util.*;!import com.google.gson.Gson;!import com.google.gson.GsonBuilder;!!public class HelloWorld {! !public static void main(String[] args) throws Exception {!

 !List<URI> hosts = Arrays.asList(!new URI("http://127.0.0.1:8091/pools")!

);!!

CouchbaseClient cb = new CouchbaseClient(hosts, "default", "");!Gson json = new Gson();!!Hashtable data = new Hashtable();!data.put("jsonkey", "value");!data.put("created_at", new Date());!!cb.add("mydata", json.toJson(data))!System.out.println(cb.get("mydata"));!

!cb.shutdown();!

}!}!

JAVA

Make a Connection

var Couchbase = require('couchbase');!var cb = new Couchbase.Connection({bucket: "default"}, function(err) { });!!var data = { jsonkey: "value", created_at: new Date().toString() };!cb.add("mydata", data);!!console.log(cb.get("mydata"));!

NODEJS

<?php!// adjust these parameters to match your installation!$cb = new Couchbase("127.0.0.1:8091", "", "", "default");!!$data = array("jsonkey" => "value", ! "created_at" => date("Y-m-d H:i:s"));!!$cb->add("mydata",json_encode($data));!var_dump($cb->get("mydata"));!!?>!

PHP

OPERATIONS

Couchbase Organization

• Couchbase operates like a Key-Value Document Store - Simple Datatypes: strings, numbers, datetime (string),

boolean, and binary data (string) can be stored- Complex Datatypes: dictionaries/hashes, arrays/lists, can

be stored in JSON format (simple lists can be string based with delimiter)

- JSON is a special class of string with a specific format for encoding simple and complex data structures

• Schema is unenforced and implicit, schema changes are programmatic, done online, and can vary from Document to Document

• get  (key)  –  Retrieve  a  document  

• set  (key,  value)  –  Store  a  document,  overwrites  if  exists  

• add  (key,  value)  –  Store  a  document,  error/excepFon  if  exists  

• replace  (key,  value)  –  Store  a  document,  error/excepFon  if  doesn’t  exist  

• cas  (key,  value,  cas)  –  Compare  and  swap,  mutate  document  only  if  it  hasn’t  changed  while  execuFng  this  operaFon

Store  &  Retrieve  OperaFons

Atomic Counter OperationsThese  operaFons  are  always  executed  in  order  atomically.  !

• incr  (key)  –  Increase  an  atomic  counter  value,  default  by  1  • cb.incr(“my_counter”)  #  now  it’s  2  

• decr  (key)  –  Decrease  an  atomic  counter  value,  default  by  1  • cb.decr(“my_counter”)  #  now  it’s  1

Non-JSON OperationsYou  can  use  these  creaFvely!  !

• prepend  (key,  value)  –  prepend  exisFng  string  in  couchbase  with  value  • cb.prepend(“mykey”,  "jumped  over  the  lazy  dog")    • cb.prepend("mykey",  "the  brown  fox  ")    

• append  (key,  value)  –  append  exisFng  string  in  couchbase  with  value  • cb.append(“mykey2”,  "oranges")    • cb.append("mykey2",  "  apples  bananas")

Retrieval Operations

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Application Server

Replica Couchbase Cluster Machine

Retrieval Operations

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Application Server

Replica Couchbase Cluster Machine

get

SELECT  *  WHERE  KEY  =  “mykey”

Storage Operations - set

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Application Server

Replica Couchbase Cluster Machine

Storage Operations - set

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Application Server

Replica Couchbase Cluster Machine

set

INSERT/UPDATE  WHERE  KEY  =  “mykey”

Storage Operations - add

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Application Server

Replica Couchbase Cluster Machine

Storage Operations - add

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Application Server

Replica Couchbase Cluster Machine

add

INSERT  KEY  =  VALUE

Storage Operations - add

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Application Server

Replica Couchbase Cluster Machine

Storage Operations - add

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Application Server

Replica Couchbase Cluster Machine

add

ALREADY  EXISTS!

Storage Operations - replace

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Application Server

Replica Couchbase Cluster Machine

Storage Operations - replace

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Application Server

Replica Couchbase Cluster Machine

replace

UPDATE  SET  KEY  =  VALUE  WHERE  KEY  =  “mykey”

Storage Operations - replace

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Application Server

Replica Couchbase Cluster Machine

Storage Operations - replace

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Application Server

Replica Couchbase Cluster Machine

replace

DOESN’T  EXIST!

Consistency

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Application Server

Replica Couchbase Cluster Machine

Consistency

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Application Server

Replica Couchbase Cluster Machine

get

CONCURRENCY

Optimistic Concurrency with CAS

• Every storage operation (including touch) creates a new "CAS" value, which is just a long int

• The CAS value simply represents the current state of the Document, it's like a version number

• You can use this CAS for "Optimistic Concurrency"- value, flags, cas = get("mykey", :extended => true)- This will only succeed if the CAS matches

• replace("mykey", newvalue, :cas => cas)- If another process had changed the "mykey" document, a

new CAS will have been generated, and that replace operation will fail

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Optimistic Concurrency with CAS

Application Server 2

Application Server 1

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Optimistic Concurrency with CAS

Application Server 2

Application Server 1

CAS: 111111111

CAS: 111111111

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Optimistic Concurrency with CAS

Application Server 2

Application Server 1

CAS: 111111111

CAS: 111111111

CAS: 222222222Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Optimistic Concurrency with CAS

Application Server 2

Application Server 1

CAS: 111111111

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Optimistic Concurrency with CAS

Application Server 2

Application Server 1

CAS: 222222222

CAS: 111111111

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Optimistic Concurrency with CAS

Application Server 2

Application Server 1

CAS: 222222222

CAS: 111111111

set/add/replace with CAS

CAS  MISMATCH

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Optimistic Concurrency with CAS

Application Server 2

Application Server 1

CAS: 222222222

CAS: 111111111

set/add/replace with CAS

CAS  MISMATCH

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Optimistic Concurrency with CAS

Application Server 2

Application Server 1

CAS: 222222222

CAS: 111111111

CAS: 222222222

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Optimistic Concurrency with CAS

Application Server 2

Application Server 1

CAS: 222222222

set/add/replace with CAS

RETRY  MUTATION  WITH  UPDATED  CAS

NEW  CAS  GENERATED

CAS: 222222222

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Optimistic Concurrency with CAS

Application Server 2

Application Server 1

CAS: 222222222

set/add/replace with CAS

RETRY  MUTATION  WITH  UPDATED  CAS

NEW  CAS  GENERATED

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Optimistic Concurrency with CAS

Application Server 2

Application Server 1

CAS: 222222222

CAS: 222222222

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Optimistic Concurrency with CAS

Application Server 2

Application Server 1

CAS: 333333333

CAS: 333333333

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Pessimistic Concurrency with Lock

Application Server 2

Application Server 1

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Pessimistic Concurrency with Lock

Application Server 2

Application Server 1

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Pessimistic Concurrency with Lock

Application Server 2

Application Server 1

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Pessimistic Concurrency with Lock

Application Server 2

Application Server 1

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Pessimistic Concurrency with Lock

Application Server 2

Application Server 1

set/add/replace with CAS

UNLOCKS  AFTER  COMPLETE

NEW  CAS  GENERATED

EXPIRATIONS

Storage Operations with Expirations

• CMS Framework Cache can be configured to use Couchbase- In most frameworks this is simple, as they typically already

have memcached support!

• Create/Update the Value and Expiration - [Ruby] set/add/replace("mykey", value, :ttl => 30)- [Java] set/add/replace("mykey", 30, value)!

• Update the expiration only- [Ruby] touch("mykey", :ttl => 30)- [Java] touch("mykey", 30)!

Expiration

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Application Server

Replica Couchbase Cluster Machine

Expiration

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Application Server

Replica Couchbase Cluster Machine

set/add/replace with TTLget

EXPIRED

DELETED  ON  COMPACTION

Expiration

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Application Server

Replica Couchbase Cluster Machine

Expiration

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Application Server

Replica Couchbase Cluster Machine

set/add/replace with TTL

UPDATES  TTL

touch

DURABILITY

Using Storage with Observe

• Callback when it has been written to disk on active partition• Callback when it has been written to a replica(s)• Callback when it has been written to replica(s) disk• Observe Persisted to Disk and Replicated

• [Ruby] set/add/replace("mykey", value, :observe => {:persisted => 1, :replicated => 1})

• [Java] set/add/replace("mykey", 0, value, PersistTo.MASTER, ReplicateTo.ONE)

• Observe Replicated Only• [Ruby] set/add/replace("mykey", value,

:observe => {:replicated => 1})• [Java] set/add/replace("mykey", 0, value, ReplicateTo.ONE)

Durability Persist-To

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Application Server

Replica Couchbase Cluster Machine

Durability Persist-To

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Application Server

Replica Couchbase Cluster Machine

set/add/replace

Durability Persist-To

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Application Server

Replica Couchbase Cluster Machine

set/add/replace

Callback

Durability Replicate-To

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Application Server

Replica Couchbase Cluster Machine

Durability Replicate-To

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Application Server

Replica Couchbase Cluster Machine

set/add/replace

Durability Replicate-To

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Application Server

Replica Couchbase Cluster Machine

set/add/replace

Callback

Durability Replicate-To

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Application Server

Replica Couchbase Cluster Machine

set/add/replace

Callback

Durability Replicate-To

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Application Server

Replica Couchbase Cluster Machine

set/add/replace

Callback

DEMO

Q  &  A

Resources

Main  Resource  Portal  www.couchbase.com/communiFes  !Code  Samples  Going  through  OperaUons  www.github.com/couchbaselabs/DeveloperDay  !Couchbase  Q  &  A  www.couchbase.com/communiFes/q-­‐and-­‐a  

My  Email:  jasdeep@couchbase.com  My  TwiZer:  @scalabl3

Next Webinar: Couchbase 103 - Modeling