Embed Size (px)
DESCRIPTION
by 許立文 (lwhsu)
Citation preview
Python on FreeBSD
Li-Wen Hsu <lwhsu@{lwhsu,FreeBSD}.org>
PyCon Taiwan 2012
Who is this ↘
lwhsu: 2006 開始接觸 Python
2007 得到 FreeBSD ports commit bit 2008 Sysadm & Instructor at CS.NCTU
2009 國軍 online 2010~ 菜鳥碼農+修電腦的
另外也是管 {ftp,www,cvsup,anoncvs,
portsnap,update}.tw.freebsd.org 的
</開場白>
Python
on
FreeBSD
What is
FreeBSD?
The FreeBSD Project • FreeBSD is an advanced operating system
derived from BSD, the version of UNIX®
developed at the University of California,
Berkeley. o http://www.freebsd.org/about.html
Who’s using FreeBSD? • FreeBSD is used in the devices
from the world’s big companies: o Apple
o Blue Coat
o Cisco
o F5 Networks
o Juniper
o NetApp
o Sony’s PlayStation 3
Who’s also using FreeBSD? • FreeBSD is also used to power
some of the busiest sites on
Internet: o Apache
o Netflix
o Pixnet
o WhatsApp
o Yahoo!
o Yandex
More? • Projects based on FreeBSD
o PC-BSD
o m0n0wall
o psSense
o FreeNAS
o Debian GNU/kFreeBSD
o Gentoo/FreeBSD
FreeBSD: The Good Parts • BSD License
• GEOM
• Netgraph
• Jail
• ZFS
• DTrace
• TrustedBSD MAC
Framework
• TrustedBSD Audit
• LLVM & Clang
• Ports system
• And more…
http://www.freebsd.org/features.html
</FreeBSD>
http://wiki.freebsd.org/Myths
<FreeBSD+Python>
Origin • /usr/ports/lang/python/Makefile:
# New ports collection makefile for: python
# Date created: 08 August 1995
# Whom: jkh
"Director of Engineering of Unix Technologies" at Apple
[email protected] • Mailing list [email protected]
o FreeBSD-specific Python issues
o Founded in January 2004
o http://lists.freebsd.org/mailman/listinfo/freebsd-python
o Maintaining python and python related ports, except py-qt and
py-gtk
[email protected] • Wiki
o http://wiki.freebsd.org/Python
o Who has the “Hat”
o Action items
o Contact information
• IRC o #freebsd-python on FreeNode
Friends of [email protected]
• [email protected] o Zope for FreeBSD -- porting and maintaining
o http://lists.freebsd.org/mailman/listinfo/freebsd-zope
o Founded in April 2012
• M_FBSD.python at bbs.cs.nctu.edu.tw
[email protected] • People
o alexbl (alumni)
o clsung (Taiwan)
o lwhsu (Taiwan)
o miwi
o nivit
o perky (alumni)
o rm
o sunpoet (Taiwan)
o wen
o yzlin (Taiwan)
Install Python on FreeBSD • Via ports system
o cd /usr/ports/lang/python
o make install clean
Install Python on FreeBSD • Ports options
http://xkcd.com/303/
Compiling...
http://xkcd.com/353/
import antigravity
Install Python on FreeBSD • Via package system
o pkg_add -r python
Install Python on FreeBSD • Via package system (advanced)
o setenv PACKAGEROOT http://ftp.tw.freebsd.org
o setenv PACKAGESITE
http://ftp.tw.freebsd.org/pub/FreeBSD/ports/`un
ame -p`/packages-`uname -r | cut -d - -f 1,2 | tr
"[:upper:]" "[:lower:]"`/Latest/
o pkg_add -r python
Install Python on FreeBSD • In the (on so far) future:
• Pkgng o pkg install python
Wait, I have a question! • Q: Why compile Python myself?
• A: o Optimize for your system
o Customize build options
o Easier to hack into Python in the system
Wait, I have a question! • Q: Do I really need to compile myself
on *every* machine I want to deploy?
• A: Build once, install everywhere. o cd /usr/ports/lang/python
o make package
• /usr/ports/packages/lang/python-2.7,2.tbz
o install
• pkg_add python-2.7,2.tbz
Run Python on FreeBSD • In /usr/ports
o lang/python
• (meta package)
o lang/python24
o lang/python25
o lang/python26
o lang/python27
o lang/python31
o lang/python32
• Also in /usr/ports o lang/jython
o lang/pypy
o lang/ironpython
o devel/ipython
Run many Python versions on FreeBSD
• /usr/local/bin/python o A symbolic link to default version
o PYTHON_DEFAULT_VERSION
• Can be manually defined in /etc/make.conf
• Possible values: python{2.4,2.5,2.6,2.7,3.1,3.2}
• You can install as many different
of Python versions you want o just install via /usr/ports/lang/python*
o First installed Python version will be the default
Run many Python versions on FreeBSD
• Install modules for different python
versions o cd /usr/ports/foo/py-bar
o make PYTHON_VERSION=python2.6 install clean
o This is still work in progress, there might be some
minor errors.
• For modules install scripts in bin/
Wait, I have a question! • Q: there is XXX, why do I need ports to
install python modules? o (XXX=distutils, setuptools, distribute, distutils2, pip)
• A: o For better integration with system
o Do more complex deploy
o Ex: install web applications like reviewboard, trac
• A system only consists with python and python module
• Also apache/nginx, mod_wsgi, etc.
Develop with Python on FreeBSD
• virtualenv + Jenkins o devel/py-virtualenv
o devel/jenkins
• Use virtualenv in jenkins, make sure
clean dependency.
• Have a job to build a virtualenv that is
usable by other jobs.
Develop with Python on FreeBSD
• ZFS + jail
• Jail o Light weight virtual machine
o /etc/rc.conf
• jail_enable="YES”
• ZFS o zpool create tank ad1
o zfs create tank/jail1
Develop with Python on FreeBSD
• Create a new jail o bsdinstall jail /tank/jail1
o freebsd-update -b /tank/jail1 fetch install
o Modify /etc/rc.conf
• jail_jail1_rootdir="/tank/jail1”
• jail_jail1_hostname="jail1”
• jail_jail1_ip="192.168.xxx.xxx”
• jail_jail1_interface="lo0"
o /etc/rc.d/jail start jail1
Develop with Python on FreeBSD
• Clone a jail o zfs snapshot tank/jail1@milestone1
o zfs clone tank/jail1@milestone1 tank/jail2
o Modify /etc/rc.conf
o /etc/rc.d/jail start jail2
o Another development environment is created!
Run your Python Apps on FreeBSD
• Nothing special or different with
other Unix-like platform
Run your Python Web Apps
• GIL problem
<VirtualHost *:80>
DocumentRoot "/home/www/webroot"
ServerName python.example.com
WSGIProcessGroup pythonexample
WSGIDaemonProcess pythonexample processes=16 \
threads=128 maximum-requests=1024 \
display-name=%{GROUP}
</VirtualHost>
Make Python
module port
FreeBSD Ports • Each ``port’’ contains any patches
necessary to make the original application
source code compile and run on FreeBSD.
Installing an application is as simple as
typing make in the port directory. o http://www.freebsd.org/ports/
Inside a port • ls -1 /usr/ports/devel/py-coverage
Makefile
distinfo
pkg-descr
pkg-plist
Create a new Python port • Use porttools to help:
o cd /usr/ports/ports-mgmt/porttools
o make install clean
• Create a new port o cd /your/workspace
o port create py-newport
Makefile # New ports collection makefile for: py-coverage
# Date created: Jun. 22, 2009
# Whom: Li-Wen Hsu <[email protected]>
#
# $FreeBSD: ports/devel/py-coverage/Makefile,v 1.9 2012/05/19 15:15:11 lwhsu Exp $
#
PORTNAME= coverage
PORTVERSION= 3.5.2
CATEGORIES= devel python
MASTER_SITES= CHEESESHOP
PKGNAMEPREFIX= ${PYTHON_PKGNAMEPREFIX}
MAINTAINER= [email protected]
COMMENT= Code coverage measurement for Python
LICENSE= BSD
USE_PYTHON= yes
USE_PYDISTUTILS= easy_install
PYEASYINSTALL_ARCHDEP= yes
.include <bsd.port.mk>
Makefile # New ports collection makefile for: py-coverage
# Date created: Jun. 22, 2009
# Whom: Li-Wen Hsu <[email protected]>
#
# $FreeBSD: ports/devel/py-coverage/Makefile,v 1.9 2012/05/19 15:15:11
lwhsu Exp $
#
Makefile PORTNAME= coverage
PORTVERSION= 3.5.2
CATEGORIES= devel python
MASTER_SITES= CHEESESHOP
PKGNAMEPREFIX= ${PYTHON_PKGNAMEPREFIX}
Makefile LICENSE= BSD
USE_PYTHON= yes # 2.6 2.6+ -2.7 3.1+
USE_PYDISTUTILS= easy_install
PYEASYINSTALL_ARCHDEP= yes
.include <bsd.port.mk>
distinfo SHA256 (coverage-3.5.2.tar.gz) =
b9e0dc65e42236a3c9b8978701bc9e7298ed7e24d39f99
ea2ba4b945d537fe55
SIZE (coverage-3.5.2.tar.gz) = 115497
pkg-descr Coverage measures code coverage, typically during test execution.
It uses the code analysis tools and tracing hooks provided in the
Python standard library to determine which lines are executable,
and which have been executed.
WWW: http://nedbatchelder.com/code/coverage
pkg-plist bin/coverage
%%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/EGG-INFO/PKG-INFO
%%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/EGG-INFO/SOURCES.txt
%%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/EGG-INFO/dependency_links.txt
%%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/EGG-INFO/entry_points.txt
%%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/EGG-INFO/native_libs.txt
%%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/EGG-INFO/not-zip-safe
%%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/EGG-INFO/top_level.txt
%%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/coverage/__init__.py
%%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/coverage/__init__.pyc
%%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/coverage/__init__.pyo
[...]
%%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/coverage/xmlreport.py
%%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/coverage/xmlreport.pyc
%%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/coverage/xmlreport.pyo
@dirrm %%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/coverage/htmlfiles
@dirrm %%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/coverage
@dirrm %%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%/EGG-INFO
@dirrm %%PYTHON_SITELIBDIR%%/%%PYEASYINSTALL_EGG%%
Before submitting • Check
o Use portlint (ports-mgmt/portlint)
o Command:portint –abct
• Test o Use porttools (ports-mgmt/porttools)
o Command: port test
Before submitting • More test
o Use Tinderbox (ports-mgmt/tinderbox)
o A comprehensive way to test ports
• And builds packages!
Wait, I have a question! • Q: Setting up a tinderbox is too complex
• A: Use RedPorts http://redports.org/
Submit! • In your port directory
• Command: o port submit
• Quit editor: o s)end,
e)dit or
a)bort? s
After submitting • A PR (problem report) is created
• A committer will handle it
• A new port is in the ports tree! o See on http://freshports.org/foo/bar
• To update your ports tree: o portsnap fetch update
• Install the new port: o cd /usr/ports/foo/bar; make install clean
Future Work of [email protected]
• Remove 2.4 and 2.5 in the ports tree o 2012-06-30
• Handle packages with pip/distribute
• Debug versions of lang/python*
• Install only unzipped eggs
• Truly modules for multiple python
versions co-exist
Q & A
