Couchbase 102 - SDK Operations

Technical Evangelist

twi0er: @scalabl3email: [email protected]

Jasdeep Jaitla

Couchbase 102: SDK OperaFons

SETUP SDK

Supported SDK'swww.couchbase.com/communi/es

• Each supported SDK page has instrucFons for setup

• PHP, Ruby, NodeJS and Python clients are wrappers around libcouchbase C library, so libcouchbase must be installed first

• For other community clients, click on "All Clients" on leS nav, scroll down the page and you can see clients for Go, Erlang, Clojure, TCL, other nodejs and Perl.

Installing Libcouchbase

• Mac Tips before Libcouchbase & SDK Install

• Make sure you have XCode & Command Line Tools Installed

• Install Homebrew if you don't have it already

• Do a $ brew update, $ brew upgrade and $ brew doctor to be sure you're up to date

www.couchbase.com/communi/es/c/geBng-‐started

Installing Libcouchbase

• Mac Via Homebrew

• $ brew install libcouchbase

• PC-‐Windows

• Download appropriate Zip from website

• Redhat/CentOS

• wget the yum repositories

• $ sudo yum install -‐y libcouchbase2-‐libevent libcouchbase-‐devel

• Ubuntu

• wget ubuntu repositories

• $ sudo apt-‐get install libcouchbase2-‐libevent libcouchbase-‐dev

www.couchbase.com/communi/es/c/geBng-‐started

Client Connections

Couchbase Server8091

11210

8092

11211Application Server

Client Connections


11210

8092


HTTP

Cluster Configura/on and Cluster Par//on Map

Client Connections


11210

8092


HTTP

Cluster Configura/on and Cluster Par//on Map

MAP

Client Connections


11210

8092


HTTP

MAP

Client Connections


11210

8092


HTTP

MAP

TCP Binary

Create Socket Connec/on to Each Node in Cluster

Client Connections


11210

8092


HTTP

MAP

TCP Binary

Client Connections


11210

8092

11211

HTTP

Application Server

HTTP

MAP

View Querying

TCP Binary

Client Connections


11210

8092

11211

HTTP

Application Server

HTTP

MAP

TCP Binary

Make a Connectionrequire 'rubygems'!require 'couchbase'! !cb = Couchbase.connect(!

:bucket => "default",!:hostname => "localhost")!!

data = { jsonkey: "value", created_at: Time.now }!!cb.add("mydata", data)!puts cb.get("mydata")

#!/usr/bin/env python!from couchbase import Couchbase!from pprint import pprint!from datetime import datetime!!cb = Couchbase.connect(bucket='default')!!data = { "jsonkey": "value", "created_at": datetime.now() }!!cb.add('mydata', data)!result = cb.get('mydata')!pprint(result.value, indent=4)!

RUBY

PYTHON

Make a Connectionimport com.couchbase.client.CouchbaseClient;!import java.net.URI;!import java.util.*;!import com.google.gson.Gson;!import com.google.gson.GsonBuilder;!!public class HelloWorld {! !public static void main(String[] args) throws Exception {!

!List<URI> hosts = Arrays.asList(!new URI("http://127.0.0.1:8091/pools")!

);!!

CouchbaseClient cb = new CouchbaseClient(hosts, "default", "");!Gson json = new Gson();!!Hashtable data = new Hashtable();!data.put("jsonkey", "value");!data.put("created_at", new Date());!!cb.add("mydata", json.toJson(data))!System.out.println(cb.get("mydata"));!

!cb.shutdown();!

}!}!

JAVA

http://127.0.0.1:8091/pools

Make a Connection

var Couchbase = require('couchbase');!var cb = new Couchbase.Connection({bucket: "default"}, function(err) { });!!var data = { jsonkey: "value", created_at: new Date().toString() };!cb.add("mydata", data);!!console.log(cb.get("mydata"));!

NODEJS

<?php!// adjust these parameters to match your installation!$cb = new Couchbase("127.0.0.1:8091", "", "", "default");!!$data = array("jsonkey" => "value", ! "created_at" => date("Y-m-d H:i:s"));!!$cb->add("mydata",json_encode($data));!var_dump($cb->get("mydata"));!!?>!

PHP

OPERATIONS

Couchbase Organization

• Couchbase operates like a Key-Value Document Store - Simple Datatypes: strings, numbers, datetime (string),

boolean, and binary data (string) can be stored- Complex Datatypes: dictionaries/hashes, arrays/lists, can

be stored in JSON format (simple lists can be string based with delimiter)

- JSON is a special class of string with a specific format for encoding simple and complex data structures

• Schema is unenforced and implicit, schema changes are programmatic, done online, and can vary from Document to Document

• get (key) – Retrieve a document

• set (key, value) – Store a document, overwrites if exists

• add (key, value) – Store a document, error/excepFon if exists

• replace (key, value) – Store a document, error/excepFon if doesn’t exist

• cas (key, value, cas) – Compare and swap, mutate document only if it hasn’t changed while execuFng this operaFon

Store & Retrieve OperaFons

Atomic Counter OperationsThese operaFons are always executed in order atomically. !

• incr (key) – Increase an atomic counter value, default by 1 • cb.incr(“my_counter”) # now it’s 2

• decr (key) – Decrease an atomic counter value, default by 1 • cb.decr(“my_counter”) # now it’s 1

Non-JSON OperationsYou can use these creaFvely! !

• prepend (key, value) – prepend exisFng string in couchbase with value • cb.prepend(“mykey”, "jumped over the lazy dog") • cb.prepend("mykey", "the brown fox ")

• append (key, value) – append exisFng string in couchbase with value • cb.append(“mykey2”, "oranges") • cb.append("mykey2", " apples bananas")

Retrieval Operations

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Application Server

Replica Couchbase Cluster Machine

Retrieval Operations

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Application Server


get

SELECT * WHERE KEY = “mykey”

Storage Operations - set

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Application Server


Storage Operations - set

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Application Server


set

INSERT/UPDATE WHERE KEY = “mykey”

Storage Operations - add

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Application Server



Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Application Server


add

INSERT KEY = VALUE


Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Application Server



Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Application Server


add

ALREADY EXISTS!

Storage Operations - replace

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Application Server



Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Application Server


replace

UPDATE SET KEY = VALUE WHERE KEY = “mykey”


Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Application Server



Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Application Server


replace

DOESN’T EXIST!

Consistency

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Application Server


Consistency

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Application Server


get

CONCURRENCY

Optimistic Concurrency with CAS

• Every storage operation (including touch) creates a new "CAS" value, which is just a long int

• The CAS value simply represents the current state of the Document, it's like a version number

• You can use this CAS for "Optimistic Concurrency"- value, flags, cas = get("mykey", :extended => true)- This will only succeed if the CAS matches

• replace("mykey", newvalue, :cas => cas)- If another process had changed the "mykey" document, a

new CAS will have been generated, and that replace operation will fail

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue


Application Server 2


Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue




CAS: 111111111

CAS: 111111111

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue




CAS: 111111111

CAS: 111111111

CAS: 222222222Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue




CAS: 111111111

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue




CAS: 222222222

CAS: 111111111

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue




CAS: 222222222

CAS: 111111111

set/add/replace with CAS

CAS MISMATCH

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue




CAS: 222222222

CAS: 111111111


CAS MISMATCH

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue




CAS: 222222222

CAS: 111111111

CAS: 222222222

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue




CAS: 222222222


RETRY MUTATION WITH UPDATED CAS

NEW CAS GENERATED

CAS: 222222222

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue




CAS: 222222222


RETRY MUTATION WITH UPDATED CAS

NEW CAS GENERATED

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue




CAS: 222222222

CAS: 222222222

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue




CAS: 333333333

CAS: 333333333

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Pessimistic Concurrency with Lock



Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue




Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue




Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue




Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue





UNLOCKS AFTER COMPLETE

NEW CAS GENERATED

EXPIRATIONS

Storage Operations with Expirations

• CMS Framework Cache can be configured to use Couchbase- In most frameworks this is simple, as they typically already

have memcached support!

• Create/Update the Value and Expiration - [Ruby] set/add/replace("mykey", value, :ttl => 30)- [Java] set/add/replace("mykey", 30, value)!

• Update the expiration only- [Ruby] touch("mykey", :ttl => 30)- [Java] touch("mykey", 30)!

Expiration

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Application Server


Expiration

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Application Server


set/add/replace with TTLget

EXPIRED

DELETED ON COMPACTION

Expiration

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Application Server


Expiration

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Application Server


set/add/replace with TTL

UPDATES TTL

touch

DURABILITY

Using Storage with Observe

• Callback when it has been written to disk on active partition• Callback when it has been written to a replica(s)• Callback when it has been written to replica(s) disk• Observe Persisted to Disk and Replicated

• [Ruby] set/add/replace("mykey", value, :observe => {:persisted => 1, :replicated => 1})

• [Java] set/add/replace("mykey", 0, value, PersistTo.MASTER, ReplicateTo.ONE)

• Observe Replicated Only• [Ruby] set/add/replace("mykey", value,

:observe => {:replicated => 1})• [Java] set/add/replace("mykey", 0, value, ReplicateTo.ONE)

Durability Persist-To

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Application Server



Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Application Server


set/add/replace


Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Application Server


set/add/replace

Callback

Durability Replicate-To

Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Application Server



Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Application Server


set/add/replace


Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Application Server


set/add/replace

Callback


Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Application Server


set/add/replace

Callback


Couchbase Server

EP EngineRAM Cache

Disk Write Queue

Replication Queue

Application Server


set/add/replace

Callback

DEMO

Q & A

Resources

Main Resource Portal www.couchbase.com/communiFes !Code Samples Going through OperaUons www.github.com/couchbaselabs/DeveloperDay !Couchbase Q & A www.couchbase.com/communiFes/q-‐and-‐a

My Email: [email protected] My TwiZer: @scalabl3

Next Webinar: Couchbase 103 - Modeling

Technology

Couchbase 102 - SDK Operations