Plagger the duct tape of internet

Plaggerthe duct tape of the Web

Tatsuhiko Miyagawa miyagawa@gmail.com

Six Apart, Ltd. / Shibuya Perl MongersXML Developers' Day #9

Tatsuhiko MiyagawaTatsuhiko Miyagawa 2006/11/24 XML developers' day #92006/11/24 XML developers' day #9

IRC#plagger-ja

chat.freenode.net(iso-2022-jp)

アウェイっぽいので自己紹介

Tatsuhiko Miyagawa

http://www.vox.com/

What is Plagger?

PluggableRSS/Atom

Aggregator

Why Pluggable?

Just for a feed aggregation?

History

2002 Apr.baseball2rss

http://search.cpan.org/dist/WWW-Baseball-NPB/

2003 Oct.rss2javascript

http://blog.bulknews.net/cookbook/blosxom/rss/rss2js.html

2004 Sep.bloglines2ipod

http://bulknews.net/lib/utils/bloglines2ipod/

2004 Oct.rss2audiobook

http://bulknews.net/lib/utils/rss2audiobook/

2005 Aug.bloglines2gmail

http://svn.bulknews.net/repos/public/bloglines2email/trunk/

#!/usr/bin/perluse strict;use warnings;use LWP::UserAgent;use XML::RSS;

my $url = "http://example.com/rss.xml";my $agent = LWP::UserAgent->new;my $xml = $agent->get($url)->content;my $rss = XML::RSS->new; $rss->parse($xml);

for my $item (@{$rss->items}) { # do something with $item}

my $url = "http://example.com/rss.xml";my $agent = LWP::UserAgent->new;my $xml = $agent->get($url)->content;my $rss = XML::RSS->new; $rss->parse($xml);

"HTTP リクエストがエラーになったら ?"

my $url = "http://example.com/rss.xml";my $agent = LWP::UserAgent->new;my $res = $agent->get($url);if ($res->is_error) { die "Bah." }my $xml = $res->content;my $rss = XML::RSS->new; $rss->parse($xml);

"Atom も読みたい !"

#!/usr/bin/perluse strict;use warnings;use LWP::UserAgent;use XML::RSS;use XML::Atom::Feed;

my $url = "http://example.com/rss.xml";my $agent = LWP::UserAgent->new;my $res = $agent->get($url);if ($res->is_error) { die "Bah." }my $xml = $res->content;if ($res->content_type =~ /atom/) { my $feed = XML::Atom::Feed->new(\$xml);} else { my $rss = XML::RSS->new; $rss->parse($xml);}

If-Modified-Since で帯域節約したい！

my $url = "http://example.com/rss.xml";my $agent = LWP::UserAgent->new;my $local = cache_path_for($url);my $res = $agent->mirror($url, $local);if ($res->is_error) { die "Bah." }my $xml = $res->content;…

" 壊れたフィードもパースしたい！ "

Etc., etc.

他にも

rss2opml

http://aruntx.com/software/rss2opml/

rss2pdf

http://rss2pdf.com/

rss2atom

brian.wanamaker.com/mybicycle/2004/02/rss2atom.html

atom2rss

http://www.2rss.com/software.php?page=atom2rss

rss2ical

http://bura-bura.com/blog/archives/2004/06/22/rss2ical/

Bloglines2opml

http://mycvs.org/wp/wp-content/wp-transform.php

rss2gmail

http://www.cs.utexas.edu/~karu/gmailrss/

rss2imap

http://rss2imap.sourceforge.jp/

ebay2rss

http://www.2rss.com/software.php?page=ebay2rss

svn2rss

http://twiki.org/cgi-bin/view/Codev/Svn2rss

Where either of <any> is RSS|Atom|OPML

This is ridiculous.

Different Languages,Different Bugs.No hackability

via http://www.atmarkit.co.jp/fnetwork/rensai/5minplagger/02.html

IRC, Eject, GrowlMSAgent, SSTP …

Filter Publish

StripRSSAdTruePermalinkEntryFullTextPipeThumbnailFindEnclosuresFetchEnclosureSpamAssassinRSSLiberalDateTimeURLBLResolveRelativeLink…

GmailDeliciousPDFMTFeedPlanetSpeech…

Notify

BloglinesConfigOPML, XOXOFile, DBI, FOAF …

Mixi, FrepaPOP3, iCaliTunes, AmazonYouTube …

Subscription

CustomFeed

Filter Publish

Notify

Subscription

CustomFeed

Filter Publish

Notify

Subscription

CustomFeed

Filter Publish

Notify

Subscription

CustomFeed

Just like UNIX pipe

Subscribe OPML | StripRSSAd | ResolveRelativeLink \| Publish Feed --type=Atom

I believe RSS has the potential to be the “UNIX pipe of the internet” …

Ray OzzieCTO of Microsoft

http://rayozzie.spaces.live.com/blog/cns!FB3017FBB9B2E142!285.entry

"the Unix shell for Web 2.0"

組み合わせの数

{CustomFeed,Subscription}/*.pm: 35

{Publish,Notify}/*.pm: 37

35 * 37 = 1295

Plagger Core features

RSS/AtomAuto-Discovery

Support Feed formatsRSS 0.91 to Atom 1.0(XML::Feed + hacks)

Support parsingBroken feeds(XML::Liberal)

HTTP optimizationsIf-Modified-Since /

gzip(URI::Fetch)

Podcast / VideocastSupport

(RSS 2.0 & Atom 1.0)

PhotocastMedia RSS

iTunes RSS*

非同期ダウンロードcURL, wget,

HTTP::Parallel & HTTP::Async*

完全な国際化Unicode & Timezone

Access tobrowser Cookies

IE, Safari, Firefox and w3m

Screen-scrapingVia CutomFeed::*

Stackable Plugins

Rule-basedDispatch of Plugins

Plagger のインストール

省略http://plagger.org/trac/wiki/PlaggerQuickStart

Plaggerクイックチュートリアル

Email クライアントでRSS フィードを読みたい！(Gmail, Thunderbird)

rss2email.yaml

plugins: - module: Subscription::Config config: feed: - http://bulknews.vox.com/library/posts/atom.xml - http://bulknews.typepad.com/blog/ - module: Publish::Gmail config: mailto: miyagawa@gmail.com

RSS in Gmail

HTML + Images

Feed Image (Logo / Buddy Icon)

Search

Auto grouping (“Conversations”)

オフライン（飛行機）でもフィードを読みたい！

rss2email.yaml

plugins: - module: Subscription::Config config: feed: - http://bulknews.vox.com/library/posts/atom.xml - http://bulknews.typepad.com/blog/ - module: Filter::FindEnclosures - module: Filter::FetchEnclosure config: dir: /tmp - module: Publish::Gmail config: mailto: miyagawa@gmail.com attach_enclosures: 1

Offline Mode

POP3 + Thunderbird

" フィードのリストをYAML で管理するのメンドウ "

opml2email.yamlplugins: - module: Subscription::OPML config: url: http://example.com/subscription.opml - module: Publish::Gmail config: mailto: miyagawa@gmail.com

# subscription.opml<?xml version="1.0"?><opml><outline title="Subscriptions"> <outline title="miyagawa" type="rss" xmlUrl="http://bulknews.typepad.com/blog/atom.xml" /> <outline title="miyagawa on Vox" type="rss" htmlUrl="http://bulknews.vox.com/" /></outline></opml>

"OPML を手動で編集するのはテラダルス "

filesub2email.yaml

plugins: - module: Subscription::File config: url: file:///path/to/subscription.txt - module: Publish::Gmail config: mailto: miyagawa@gmail.com

> cat subscription.txthttp://bulknews.typepad.com/blog/atom.xmlhttp://bulknews.vox.com/>

Tatsuhiko MiyagawaTatsuhiko Miyagawa 2006/11/23 XML Developers' Day #92006/11/23 XML Developers' Day #9

"1000 フィード以上読んでると自分のマシンで読むのは帯域のムダ。 Bloglines にクロールさせたい。 "

bloglines2email.yaml

plugins: - module: Subscription::Bloglines config: username: YOU@example.com password: blahblahblah - module: Publish::Gmail config: mailto: miyagawa@gmail.com

" それ Livedoor Reader で "

plugins: - module: Subscription::LivedoorReader config: username: YOU@example.com password: blahblahblah - module: Publish::Gmail config: mailto: miyagawa@gmail.com

"Gmail / Thunderbird からdel.icio.us にブクマしたい !"

plugins: - module: Subscription::Bloglines config: username: YOU@example.com password: blahblahblah

- module: Widget::Simple config: widget: delicious

- module: Publish::Gmail config: mailto: miyagawa@gmail.com

" それはてなブックマークで "

- module: Widget::Simple config: widget: hatena_bookmark_users

"RSS 広告ウザス "

- module: Widget::Simple config: widget: delicious

- module: Filter::StripRSSAd

Quick tourFor more plugins

Plugin phases (types)

SubscriptionAggregatorCustomFeed

FilterPublishSearchNotify

Subscription

load subscriptions(list the feeds/URLs to aggregate)

Subscription::Config - module: Subscription::Config config: feed: - http://www.yapcchicago.org/feed/ - http://tokyo.yapcasia.org/blog/

- module: Subscription::OPML config: url: http://www.example.com/subs.opml

# subs.opml <opml> <outline xmlUrl="http://www.yapcchicago.org/feed/" /> <outline htmlUrl="http://tokyo.yapcasia.org/blog/" /> </opml>

Subscription::OPML

- module: Subscription::File config: url: file:///path/to/subscription.txt

% cat subscription.txt http://www.yapcchicago.org/feed/ http://tokyo.yapcasia.org/blog/ %

Subscription::File

Subscription::XOXO - module: Subscription::XOXO config: url: http://www.example.com/subscription.html

# subscription.html <ul class="xoxo"> <li><a href="http://www.yapcchicago.org/feed/">YAPC::NA</a></li> <li><a href="http://tokyo.yapcasia.org/blog/">YAPC::NA</a></li> </ul>

Subscription::Bookmarks

Read bookmarks file of IE, Firefox and Safari

Filter

Normalize / Repair feed metadataUpgrade feed content

Filter feed content using text filtersInvoke some action on entries

Filter::EntryFullText

本文なしのフィードをアップグレード個別 HTML を取得して正規表現 / XPath

Filter::TruePermalink

リダイレクト URL などを Canonicalize(e.g. http://…/go.php?url=….)

Filter::FindEnclosures

コンテンツからエンクロージャを抽出<a href="http://…./foo.mp3">episode

#1</a>

Publish

Publish aggregated entry to online servicesreBlogging

Convert feeds to other formats

Publish::Feed

Republish feed in RSS/AtomGood to use with scrapers

Publish::MT

Reblog entries using MT XML-RPC

Publish::MTWidget

Publish::Email

text/plain, multipart/alternativePluggable email protocols

(SMTP, SMTP Auth, IMAP, Maildir …)

Publish::iCal

Publish iCal feeds out of RSS/Atom

Publish::Excel

仕事の合間に !

Search

Index aggregated entries on search engines

Search::Spotlight

Search::Estraier

Uses HyperEstraier XMLRPC node API

Search::Lucene*

Use Lucene WebService API (OpenSearch 1.1 and Atom)

Notify

Notify feed updates in various ways

Notify::Growl

Notify::Balloon

Notify::MSAgent

Notify::Eject

Supports: Windows, Linux, FreeBSD and Mac OSX!

Notify::Pizza

Now it does Sushi too!

Plagger に対する誤解

それ Plagger でできるよ

それプラズマで説明できるよ

http://d.hatena.ne.jp/sugarcut/20061117/p1

それ Pla 脳

何でもできる

… わけじゃない

Plagger is a"Feed" aggregator

Plagger::Subscription

Plagger::Feed

authoridlinktitletagsurlentries

Plagger::Feed

Plagger::Entry

authoridlinkpermalinktitletagsenclosures

ピザ Pla の意義

"the Unix shell for Web 2.0"

"RSS is the Standard IO for the Web"

Feed formats

RSS 0.91RSS 2.0RSS1.0 / RDF

Atom 1.0

XOXOXBEL

Sitemaps

attention.xml

Amazon API

Google APIOpenSearch

AtomPP

Enclosures

Feed Vocabulary / Extensions

Photocast

iTunes RSSMedia RSS

Dublin Core

microformats

Plagger = The duct tape of the web.

Example:Location metadata

N 37.7782 W 122.3973

GeoRSS

<georss:point>37.7782 -122.3973</georss:point>

<georss:where> <gml:Point> <gml:pos>37.7782 -122.3973</gml:pos> </gml:Point></georss:where>

xmlns:georss="http://www.georss.org/georss"xmlns:gml="http://www.opengis.net/gml"

RDF geo vocabulary

<foaf:based_near> <geo:Point> <geo:lat>35.678</geo:lat> <geo:long>139.770</geo:long> </geo:Point></foaf:based_near>

xmlns:geo="http://www.w3.org/2003/01/geo/wgs84_pos#"

Flickr geo tag

<media:category scheme="urn:flickr:tags">geo:lat=37.7782 geo:lon=-122.3973</media:category>

xmlns:media="http://search.yahoo.com/mrss/"

GeoURL

geo microformats

Links to Google Maps

<a href="http://maps.google.com/maps?q=37.7782,-122.3973&z=16">Link to Google Maps</a>

Eznavi mail

Photo EXIF

<div class="adr"> <div class="street-address">548 4th St.</div> <span class="locality">San Francisco</span>, <span class="region">CA</span> <span class="postal-code">94107</span> <div class="country-name">U.S.A.</div></div>

We don't careabout format diffs.

GeoRSS

<georss:point>37.7782 -122.3973</georss:point>

<georss:where> <gml:Point> <gml:pos>37.7782 -122.3973</gml:pos> </gml:Point></georss:where>

Namespace::GeoRSS

RDF geo vocabulary

<foaf:based_near> <geo:Point> <geo:lat>35.678</geo:lat> <geo:long>139.770</geo:long> </geo:Point></foaf:based_near>

Namespace::Geo

Flickr geo tag

<media:category scheme="urn:flickr:tags">geo:lat=37.7782 geo:lon=-122.3973</media:category>

Filter::geotagged

GeoURL

Filter::GeoURL

geo microformats

Filter::Microformats::geo

Links to Google Maps

<a href="http://maps.google.com/maps?q=37.7782,-122.3973&z=16">Link to Google Maps</a>

Filter::ExtractMapsLinks

Eznavi mail

Filter::ExtractMapsLinks

Photo EXIF

Filter::FetchEnclosure +Filter::ExtractEXIF

<div class="adr"> <div class="street-address">548 4th St.</div> <span class="locality">San Francisco</span>, <span class="region">CA</span> <span class="postal-code">94107</span> <div class="country-name">U.S.A.</div></div>

Filter::Microformats::adr +Filter::Geocoding::US

Publish::KMLPublish::GoogleMaps

Publish::Feed(with geotags)

Everything's donein plugins

= Clean & extensible.

Plaggerdev. Status

Version0.7.13

Coming Soon …

iTunes RSS support

Geo extensions

Enclosure processors

ffmpeg, Sync::PSP, Sync::iPodVideo

Pluggable summarizer

& text formatter

Lingua::EN::Summarize, Text::Original, HTML::WikiConverter, HTML::FormatText

Rich Media metadata

ID3 tag in enclosuresLinks to imdb.com / amazon.com

hReview microformats

Calendar Support

iCal parser & emitterhCalendar microformats.ics attached in emails

Sync::SyncML

Email refactoring

text/plain, iso-2022-jp supportPluggable storage engines

http://plagger.org/

Planet, Mailing List, IRCBug Tracking, SVN repository

#plagger-ja on freenode

Join Us!

Thank you

Questions?

Plagger the duct tape of internet

Technology

Bile duct injuries during laparoscopic cholecystectomy · Bile duct injuries during laparoscopic cholecystectomy ... Bile duct injuries during laparoscopic cholecystectomy ... Rua

Tras Duct Ores

Duct Calculator (Revisi-1)

Re Duct Ores

9I: PAGIGING MALIKHAIN SA MGA UPUAN AT MGA SUPLAY · ⇒ Patong-patong na mga magasin (naka-duct tape nang magkakasama) mga puzzle board, foam mat, o anumang manipis, solido at mapagpapatong-patong

Round Duct System - SCHAKO

AC Duct Cal

Laporan Abp Ragi Tape Dan Tape

Spiral Duct - 栗本鐵工所. 26 KURIMOTO SPIRAL DUCT スパイラルダクト 3 KURIMOTO STAINLESS DUCT ステンレスダクト 11 SLIDE SPIRAL DUCT スライドスパイラルダクト

Date tehnice Duct ventiloconvectoaregree-hvac.ro/.../02/date-tehnice-duct-ventiloconvectoare.pdfDate tehnice Duct ventiloconvectoare 2 Circuite Air flow volume(H/M/L) CFM pa PhN/Hz

Faten presentation- Duct Tape Marketing Session 6

Sales Network - zh-cn.symbioinc.com.twº”金型錄.pdf · POLYVINYL CHLORIDE(PVC) Insulation Tape 2 Double-sided Tape 10 DOUBLE-SIDED TAPE Anti-slip Tape 14 Kraft Paper Tape 15

7 Pasos Para El Exito en Marketing Duct Tape Marketing

„Talente“ – der Internationale Gestalter-Nachwuchs in München7360.pdf · Eisen, duct tape, Farbe, ge-schnitten, PUK geschweißt; 55x28x7 cm . Insgesamt ist der Schmuck durch

Recommended Duct Catalogue

Presentasi First Duct

LSC BUS DUCT - LS-VINA Cablelsvinacable.com.vn/.../1/LS_Busduct_06020.pdf · LSC BUS DUCT LS Cable Bus Duct System BUSDUCT최종(P) ... This efficient design allow power to ... LSC

med.mahidol.ac.thmed.mahidol.ac.th/surgery/sites/default/files/public/pdf/(Update).pdf · Figure 3 Anatomy of bile duct ... RHD = Rt. Hepatic duct, CHD = Common hepatic duct -Vascular

Duct Tape Rebrand

50417982 Presentasi First Duct