Click here to load reader
Upload
crossref
View
271
Download
0
Embed Size (px)
Citation preview
guarding your data
Andrew Gilmartin senior developer [email protected]
don’t loose the data
• Oracle holds the data in Boston
Oracle × 2 + tape + off-site tape
• Oracle holds the data in Denver
Oracle × 2
• Denver less than 20 minutes out of sync with Boston
syncing Boston & Denver
• Oracle's
Oracle Streams …
• Other's
Tungsten Replicator …
• Our's
the tool’s context
• use existing tooling
• use a small set of dependent parts
• use already monitored parts
• Oracle
• file systems
• networks
• …
the tool’s necessities
• reliable
• recoverable
• chatty
• alarming
Oracle’s archive-log
1 archive-log represent the changes to the database since the last archive-log
2 discover created archive-log via Boston Oracle’s tracing-log
3 transfer archive-log to Denver
4 register archive-log with Denver’s Oracle,
5 discover archive-log applied via Denver Oracle’s tracing-log
a bash script
• script & configuration are short at ~200 & ~10 lines respectively
• same script & configuration in Boston & in Denver
• Boston based script relies on Denver based script and vise versa
• a known set of functions
significant event markers
• events triggered by tracing-log files
• events are recorded & coordinated via marker files
• events are created, transferred, registered, & applied
• markers are distinctively named, empty files in Boston
arch_1_640475_754538743.arc.created
archive-log’s name! marker!
were what happens
• Boston continuously watches for archive-log creations
invokes “transfer & register” process in Denver
• Denver continuously watches for archive-log applications
invokes “applied” process in Boston
• Boston periodically investigate process’s status
• Boston & Denver periodically clean up
some implementation notes
• humans are are good at recovery
• use the operating system ... ssh keys ... directory as database
• use configuration files rather than command line options
• Use command timeouts
• check command exit status
• retry commands at lengthening intervals … 5s 30s 300s …
• use bash's -E option
some resources
• spark.sh & spark.conf
• ask Chuck for our tool, [email protected]
• Giuseppe Maxia’s “Script it! Make professional DBA tools out of nothing” https://www.percona.com/live/mysql-conference-2013/sessions/script-it-make-professional-dba-tools-out-nothing