Upload
takayuki-shimizukawa
View
610
Download
6
Embed Size (px)
Citation preview
Easy contributable internationalization process with Sphinx
Takayuki Shimizukawa
Sphinx co-maintainer
Sphinx-users.jp1
안녕하세요 .annyeonghaseyo.나는시미즈카와입니다 .
naneun shimizukawa imnida.
2
こんにちはKon-Nichi-Wa
3
Who am I
@shimizukawa
Python developer since 2004
Sphinx co-maintainerSphinx-users.jp organizerPyCon JP committeeBePROUD co, ltd. member
4
Agenda
1. Sphinx introduction & Setup for i18n
2. A easy way and Non easy ways to translate
3. Sphinx i18n feature
4. Automated translation process with several services
5. Tips6
Question
7
Question1
How many people do you use Open Source
Software?
8
Question2
How many people may have contributed to the
OSS?
9
Question3
Does the translation into familiar language
contribute to other developers?
10
11
Translator
Product Author
Translated Docs
Readers
English Docs
Translators
Tools & Services
Tools1. sphinx - documentation generator2. docutils - base of sphinx3. sphinx-intl - support utility for sphinx i18n4. transifex-client - file transfer tool for
Transifex
Services5. Transifex - translation support service6. Drone.io - continuous integration service
12
1. Sphinx introduction& Setup for i18n
13
What is Sphinx?
Sphinx is a documentation generator
Sphinx generates doc as several output formats from the reST text
markup
14
1. Sphinx introduction & Setup for i18n
Sphinx
reSTreSTreStructuredText
(reST)reST
Parser
HTML Builder
ePub Builder
LaTeX Builder texlive
HTMLtheme
Favorite Editor
The history of Sphinx (short ver.)
15
1. Sphinx introduction & Setup for i18n
The father of Sphinx
Too hard to maintenance
~2007
Easy to writeEasy to
maintenance2007~
Sphinx Before and After
Before
There was no standard ways to write documentsSometime, we need converting markups into other
formats
After
Generate multiple output format from single sourceIntegrated html themes make read docs easierAPI references can be integrated into narrative docsAutomated doc building and hosting by ReadTheDocs
16
1. Sphinx introduction & Setup for i18n
Many docs are written by SphinxFor examples
Python libraries/tools:Python, Sphinx, Flask, Jinja2, Django, Pyramid, SQLAlchemy, Numpy, SciPy, scikit-learn, pandas, fabric, ansible, awscli, …
Non python libraries/tools:Chef, CakePHP(2.x), MathJax, Selenium, Varnish
17
1. Sphinx introduction & Setup for i18n
Many docs are written by SphinxFor examples
Python libraries/tools:Python, Sphinx, Flask, Jinja2, Django, Pyramid, SQLAlchemy, Numpy, SciPy, scikit-learn, pandas, fabric, ansible, awscli, …
Non python libraries/tools:Chef, CakePHP(2.x), MathJax, Selenium, Varnish
18
1. Sphinx introduction & Setup for i18n
SPHINXdocutils
HTML Builder HTML theme(Jinja2)
gettext Builder
*.pot
*.po
I18N
*.mo OmegaT
Pootle
Transifex
Translation Tools, Services
Favorite Editor
Sphinx i18n feature (built-in)
19
1. Sphinx introduction & Setup for i18n
reST Parser(directive / role)
doctree(Intermediat
e)
reSTreSTreStructuredText(reST)
$ pip install sphinx
Support tools for translation
How to install Sphinx with i18n tools
To build a sphinx document
$ pip install sphinx-intl$ pip install transifex-client
20
1. Sphinx introduction & Setup for i18n
$ git clone https://github.com/shimizukawa/deepthought.git$ cd deepthought/doc$ make html...Build finished. The HTML pages are in _build/html.
"make html" command generates html files into _build/html.
make html once
21
1. Sphinx introduction & Setup for i18n
Files & setup conf.py$ tree /path/to/deepthought+- deep_thought| +- __init__.py| +- api.py| +- calc.py| +- utils.py+- doc| +- _build/| | +- html/ | +- _static/| +- _template/| +- conf.py| +- index.py| +- make.bat| +- Makefile+- setup.py
22
1. Sphinx introduction & Setup for i18n
Document source
Document build output
Target library for doc1. ...2. 3. language = 'ko'4. locale_dirs = ['locale']5.
doc/conf.py
2. Easy contributable internationalization process with Sphinx
23
Easy contributable internationalizatio
n process with Sphinx
24
What is internationalization?
25
2. Easy contributable internationalization process with Sphinx
I18N
Easy contributable internationalization process with Sphinx
26
What is easy contributable?
27
2. Easy contributable internationalization process with Sphinx
Not a easy way (example 1/3)
The manual are provided only in the HTML files
You can rewrite HTML filesHow to follow the original?
28
2. Easy contributable internationalization process with Sphinx
Not a easy way (example 2/3)
The HTML manual are generated fromreST files and
docstrings in the source files
You can rewrite reST files and docstringsHow to follow the upstream?
29
2. Easy contributable internationalization process with Sphinx
Not a easy way (example 3/3)
OK, we are engineers. we can use GIT!
Learn gitLearn GitHubgit clone to get the codeTranslation (Yes! This is what I want to
do!)git commitgit pullSometimes you must resolve conflictgit push
30
2. Easy contributable internationalization process with Sphinx
Not easy vs easy
Not a easy way Easy way1. Learn git2. Learn GitHub3. git clone to get the
code4. Translation5. git commit, pull,
push6. Resolve conflict7. Build html your self
1. No git2. No Github3. No file4. Translation5. Update
Automatically6. No conflict7. You can get a HTML
output w/o hand-build.
31
2. Easy contributable internationalization process with Sphinx
3. Sphinx i18n feature
32
Two i18n features of SphinxOutput pot files:
from reST
Input po files:to generate translated HTML
33
3. Sphinx i18n feature
reST pot
reST
htmlpo
Translation flowGenerate pot
Translate pot into po
Generate Translated HTML
34
3. Sphinx i18n feature
reST pot
reST
htmlpo
pot poTranslator
Pleasetranslate
Translate!Thanks for yourContribution!
#: ../../../deep_thought/utils.py:docstring of deep_thought.utils.dumps:1msgid "Serialize ``obj`` to a JSON formatted :class:`str`."msgstr ""
msgid "For example:"msgstr ""
35
Output pot files 3. Sphinx i18n feature
$ make gettext...Build finished. The message catalogs are in _build/gettext.
$ ls _build/gettextapi.pot examples.pot generated.pot index.pot
generated.pot
reST pot
Preparing po files to translate
doc+- _build/| +- gettext/| +- api.pot| +- examples.pot| +- generated.pot| +- index.pot+- locale/
doc+- _build/+- locale/ +- ko/ | +- LC_MESSAGES/ | +- api.po | +- examples.po | +- generated.po | +- index.po +- ja/
copy and renamefor each langs
Translate them
Translator
pot po
36
3. Sphinx i18n feature
37
Preparing po files to translate
$ sphinx-intl update -p _build/gettext -l koCreate: locale/ko/LC_MESSAGES/api.poCreate: locale/ko/LC_MESSAGES/examples.poCreate: locale/ko/LC_MESSAGES/generated.poCreate: locale/ko/LC_MESSAGES/index.po
At first, sphinx-intl copy pot files and rename them
pot po
sphinx-intl
$ make gettext$ sphinx-intl update -p _build/gettext -l koNot Changed: locale/ko/LC_MESSAGES/api.poUpdated: locale/ko/LC_MESSAGES/examples.po +3, -1Not Changed: locale/ma/LC_MESSAGES/generated.po
After change the document, sphinx-intl update differences
3. Sphinx i18n feature
Translate po files
#: ../../../deep_thought/utils.py:docstring of deep_thought.utils.dumps:1msgid "Serialize ``obj`` to a JSON formatted :class:`str`."msgstr ""
generated.po
pot po
sphinx-intl
Translator
#: ../../../deep_thought/utils.py:docstring of deep_thought.utils.dumps:1msgid "Serialize ``obj`` to a JSON formatted :class:`str`."msgstr "JSON 형식의 :class:`str` 유형에 ``obj`` 를 직렬화 ."
generated.po
Translate by using Vim, Emacs, OmegaT, ...
38
3. Sphinx i18n feature
Input po files
39
3. Sphinx i18n feature
reST
htmlpo
Translated
$ make html...Build finished. The HTML pages are in _build/html.
Big picture
40
3. Sphinx i18n feature
reST pot
html
po
make gettext
sphinx-intl
Translator
make html
Doc Author
TranslationCatalog
Translatedcatalog
4. Automated translation process with several services
41
Entire process to translate sphinx docs
42
4. Automated translation process with several services
reST pot
html
po
make gettext
sphinx-intl
make html
Doc Author
TranslationCatalog
Translatedcatalog
Translator
Translator
Translator
Autor / Translator
upload
Translator
clone
Translator
To Be
43
4. Automated translation process with several services
reST pot
html
po
make gettext
sphinx-intl
make html
Doc Author
TranslationCatalog
Translatedcatalog
upload
Translators
To BeAutomated
To BeParallel
clone
Translation tool typesVim / Emacs (Editors)
Edit local filesTranslation support plugins are available.
OmegaT (Translation Tools)Edit local files.Translation support features.
Transifex (Services)Edit onlineTranslation support
features.
44
4. Automated translation process with several services
po
Translators
To BeParallel
Be Parallel by using Transifex
Transifex provides...As API:Upload potDownload po
As Web console:GlossaryTranslation memoryRecommendationAuto-translation
45
4. Automated translation process with several services
po
Translators
Parallel
pot
Upload
pot
Auto Update
sphinx-intltransifex-client
po transifex-clientDownload
Translation on Transifex web console
46
4. Automated translation process with several services
Original Text
Translated Text(you should keep reST syntax)
Suggestions fromTranslation Memory (TM)
Original(pot)
Translation(po)
Copy orig to translationMachine translation
SaveReview (if needed)
Translators
Parallel1
2
4
3
56
To Be Automated
47
4. Automated translation process with several services
po
Translators
Parallel
pot
Upload
pot
Auto Update
sphinx-intltransifex-client
po transifex-clientDownload
reST
html
make gettext
make html
Doc Author
upload
To BeAutomated
clone
To Be Automated
48
4. Automated translation process with several services
pot
Upload sphinx-intltransifex-client
po transifex-clientDownload
reST
html
make gettext
make htmlupload
clone1
2 3
456
To BeAutomated
The procedure for automation
1. clone repository2. make gettext3. Upload pot4. Download po5. make html6. Upload html
49
4. Automated translation process with several services
pot
Upload sphinx-intltransifex-client
po transifex-clientDownload
reST
html
make gettext
make htmlupload
clone1
2 3
456
To BeAutomate
d
1. pip install sphinx sphinx-intl transifex-client2. git clone https://github.com/shimizukawa/deepthought.git3. cd deepthought/doc4. sphinx-intl create-transifexrc # create ~/.transifexrc5. sphinx-intl create-txconfig # create .tx/config6. make gettext7. sphinx-intl -p _build/gettext update-txconfig-resources
# update .tx/config8. tx push -s # push pot files to
transifex9. tx pull -l ko # pull po files from
transifex10. make html SPHINXOPTS="-D language=ko"
Shell commands for automationrun.sh
$ export SPHINXINTL_TRANSIFEX_USERNAME=mice$ export SPHINXINTL_TRANSIFEX_PASSWORD=42$ export SPHINXINTL_TRANSIFEX_PROJECT_NAME=deepthought-0_7$ export SPHINXINTL_POT_DIR=_build/gettext$ run.sh
50
4. Automated translation process with several services
The drone.io
51
WebHook
Deploy
Clone repositoryRun shell script
The drone.io is a continuous integration service.
4. Automated translation process with several services
GitHub + drone.io + S3
52
GitHub
Amazon S3
Transifex 1. Clone repository2. make gettext.3. Upload pot4. Download po5. make html6. Upload html
4. Automated translation process with several services
2
1
3
Be Automated
53
pot
Upload sphinx-intltransifex-client
po transifex-clientDownload
reST
html
make gettext
make htmlupload
clone1
2 3
456
To BeAutomated
UploadpotDownload
poupload
WebHookclone
1
5
6make html
make gettext2
3
4
1WebHook
Automated
4. Automated translation process with several services
Automated by drone.io
54
UploadpotDownload
poupload
WebHookclone
1
56
make html
make gettext2
3
4
1WebHook
Automated
4. Automated translation process with several services
View from doc author
Doc Author doesn't require annoying procedure.
UpdateTranslation Source
Doc Author
Notify
See
Commit
55
4. Automated translation process with several services
View from doc translatorsNo gitNo fileNo conflictUpdate AutomaticallyThey can get a translated HTML output w/o hand-
build.
Translators
ParallelSee
TranslateTranslated Pages
56
4. Automated translation process with several services
The entire automated process
57
UpdateTranslation Source
Doc Author
Translators
Parallel
Notify
See
See
Publish
Translate
Commit
DownloadTranslations
Notify
Automated
4. Automated translation process with several services
Summary
Tools1. sphinx - documentation generator2. docutils - base of sphinx3. sphinx-intl - support utility for sphinx i18n4. transifex-client - file transfer tool for
Transifex
Services5. Transifex - translation support service6. Drone.io - continuous integration service
58
4. Automated translation process with several services
5. Tips
59
msgid "Sphinx translates a set of reStructuredText_ source filesinto various output formats."msgstr "Sphinx translates a set of ““`reStructuredText <http://docutils.sphinx-users.jp/web/rst.html>`_ ““source files into various output formats."
*.po
How to handle URLs
60
5. TIPS
Sphinx translates a set of reStructuredText_ source files intovarious output formats.
.. _reStructuredText: http://docutils.sourceforge.net/rst.html
*.rstHyperlink
Target
Hyperlink Target doesn’t appear in POT files.If you want to change URLs into translation
version, you can use Embedded URLs notation.
How to change language w/o rewriting conf.py
You can use SPHINXOPTS make optionIt also works with other targets not only
’html’.linkcheck target is useful if your translation
contains URLs. It will find typos in translation text.
61
5. TIPS
$ make linkcheck SPHINXOPTS="-D language=ko"
$ make html SPHINXOPTS="-D language=ko"
You can control pot file resolutionUse gettext_compat
By default ‘gettext_compat = True’.reST files under sub directories are collected
into single POT file.
62
5. TIPS
language = 'ko'locale_dirs = ['locale']gettext_compat = False # generate .pot for each .rst
doc/conf.py
Translatable sphinx html templeteSphinx HTML template is Jinja2.You can use “{% trans %}” template tag.
“make gettext” exports “trans” contents into sphinx.pot file files.
63
5. TIPS
<p> <em>{%trans%}What users say:{%endtrans%}</em></p><p> {%trans%}“Cheers for a great tool that actually makes programmers <b>want</b>to write documentation!“{%endtrans%}</p>
_templates/index.html
Example projects
Sphinx-1.4 doc for "ja" translationhttps://drone.io/bitbucket.org/shimizukawa/sphinx-doc15/admin
Docutils doc for "ja" translationhttps://drone.io/bitbucket.org/sphinxjp/docutils-translation/admin
64
5. Tips
Questions?@shimizukawa
Grab me after the session :)
I’ll be in SPRINT venue too!
65
Thanks :)
66