Ruby utility for YQL storage

motivation:

  • I want to be able to interact w/ YQL storage as easily as I can w/ SQLite on my own machine.  Ideally, I’d just like to be able to say storage.use(‘table’).set(‘foo’, ‘bar’) and forget about it.

overview:

  • This class is based on the SQLite utility class, YQL utility function, and simple key/val layer for YQL storage I recently posted.  The methods available are use(), set(), and get().  The use() method accepts the select, update, and execute addresses of a YQL storage record.  Calling get() or set() fires off a request to read or write, respectively, a key using the YQL key/val table.

requirements/environment:

  • An “installation” (it’s just a couple static files on your server) of the YQL key/val table mentioned above
  • All other requirements are the same as for the SQLite class & YQL fn mentioned above

code:

class YqlStorage
  def use(settings={})
    @settings = settings
    return self
  end
  def get(key)
    
    response = yql( %{
      use 'http://{your domain}/kv.xml' as kv;
      select * from kv where key = '%s' and select = '%s'
    } % [ key, @settings[:select] ] )
    
    if response['error']
      raise 'error: %s ' % response['error']['description']
    elsif !response['query']['results']
      return nil
    end
    
    return response['query']['results']['result']
  end
  
  def set(key, val)

    response = yql( %{
      insert into kv (key, val) values ('%s', '%s')
    } % [ key, val ], { 'env' => 'http://{your domain}/kv.env' } )
    
    if response['error']
      raise 'YQL error: %s ' % response['error']['description']
    end
    
    return response
  end
end

usage:

  1. Save the code below into a file
  2. Edit the file to change all occurrences of ‘{your domain}’ to your domain
  3. Use (ha!) the YQL storage addresses defined in the key/val table setup for the use() settings
  4. here’s some example code
    store = YqlStorage.new.use( {
      # get these from YQL: http://developer.yahoo.com/yql/console/#h=desc%20yql.storage.admin
      :execute => 'store://h5Y4iRockdwzZdEHvGbBkCe',
      :select => 'store://deGTN05aNePaper04EOL30W',
      :update => 'store://qG4Scissors8917SHDjv88Wb'
    } )
    store.set( 'foo', 'bar' )
    p store.get( 'foo' )
    

Please let me know if you’ve got any suggestions/questions.  

And now, to lighten the mood, here’s a picture of a squirrel yawning:
Squirrel yawning
Photo credit: _temaki_

initial attempt at simple key/val layer on YQL storage

motivation:

  • YQL storage is very nice.  It’s globally distributed, performant, and free(!), but if you want to say “get me value where key = ‘foo'”, you need to apply a layer to map our keys to YQL’s keys.  Here’s my first attempt at this.

overview:

  • A single YQL storage record is used to store a JavaScript hash, JSON encoded.  When a value is requested, the table extracts the JSON from storage, decodes it, and applies the key passed w/ the request (we’ll let JS manage the mapping of keys to values ;).  If the request is to modify or delete the value, the updated hash is re-encoded to JSON and saved back to storage.

requirements/environement:

usage:

  1. Save the code below to a file on your server, eg kv.xml
  2. Edit the file to change all occurrences of ‘{your domain}’ to your domain
  3. Create a file called kv.env and define your table url and YQL storage addresses into it, as described in the YQL documentation
  4. For select queries, just pass the select address in the request.  For all other queries, set env={your domain}/kv.env in the request URL

code:

<?xml version="1.0"?>
<table xmlns="http://query.yahooapis.com/v1/schema/table.xsd">
    <meta>
        <author>Erik Eldridge</author>
        <documentationURL></documentationURL>
        <sampleQuery>use 'http://{your domain}/kv.xml' as kv; select * from kv where key='foo' and select='store://{select store id}'</sampleQuery>
    </meta>
    <bindings>
        <select itemPath="" produces="XML">
            <urls><url></url></urls>
            <inputs>
                <key id="key" type="xs:string" paramType="variable" required="true"/>
                <key id="select" type="xs:string" paramType="variable" required="true"/>
            </inputs>
            <execute>                
                // http://www.JSON.org/json2.js w/ alert removed
                y.include('http://{your domain}/json2.js');
                
                // supplant fn (credit: http://www.crockford.com/javascript/remedial.html)
                if(!String.prototype.supplant){String.prototype.supplant=function(o){return this.replace(/{([^{}]*)}/g,function(a,b){var r=o[b];return typeof r==='string'||typeof r==='number'?r:a;});};}
                
                // response
                response.object = function () {
                    var queries = [],
                        results = [];
                
                    queries[0] = "select * from yql.storage where name = '{select}'"
                        .supplant( { 'select' : select } );
                    results[0] = y.xmlToJson( y.query(queries[0]).results );
                    
                    if ( results[0].results.result.value ) {
                        return results[0].results.result.value[key];
                    }
                }();
            </execute>
        </select>
        <insert>
            <!-- sample query: use 'http://{your domain}/kv.xml' as kv; insert into kv (key, val) values ('foo', 'bar')
            <!-- note: use env file to define table url, select, & update -->
            <urls><url></url></urls>
            <inputs>
                <key id="select" type="xs:string" paramType="variable" required="true"/>
                <key id="update" type="xs:string" paramType="variable" required="true"/>
                <value id="key" type="xs:string" paramType="variable" required="true"/>
                <value id="val" type="xs:string" paramType="variable" required="true"/>
            </inputs>
            <execute>
                // http://www.JSON.org/json2.js w/ alert removed
                y.include('http://{your domain}/json2.js');
                
                // supplant fn (credit: http://www.crockford.com/javascript/remedial.html)
                if(!String.prototype.supplant){String.prototype.supplant=function(o){return this.replace(/{([^{}]*)}/g,function(a,b){var r=o[b];return typeof r==='string'||typeof r==='number'?r:a;});};}
                
                // response
                response.object = function () {
                    var queries = [],
                        results = [];
                    
                    queries[0] = "select * from yql.storage where name = '{select}'"
                        .supplant( { 'select' : select } );
                    results[0] = y.xmlToJson( y.query(queries[0]).results );
                    
                    results[0].results.result.value[key] = val;
                    
                    queries[1] = "update yql.storage set value = '{value}' where name = '{update}'"
                        .supplant( { 
                            'value' : JSON.stringify( results[0].results.result.value ), 
                            'update' : update
                        } );
                    results[1] = y.xmlToJson( y.query( queries[1] ).results );
                    
                    return results[1].results
                }();
            </execute>
        </insert>
        <update>
            <!-- note: use env file to define table url, select, & update -->
            <urls><url></url></urls>
            <inputs>
                <key id="select" type="xs:string" paramType="variable" required="true"/>
                <key id="update" type="xs:string" paramType="variable" required="true"/>
                <key id="key" type="xs:string" paramType="variable" required="true"/>
                <value id="val" type="xs:string" paramType="variable" required="true"/>
            </inputs>
            <execute>
                // http://www.JSON.org/json2.js w/ alert removed
                y.include('http://{your domain}/json2.js');
                
                // supplant fn (credit: http://www.crockford.com/javascript/remedial.html)
                if(!String.prototype.supplant){String.prototype.supplant=function(o){return this.replace(/{([^{}]*)}/g,function(a,b){var r=o[b];return typeof r==='string'||typeof r==='number'?r:a;});};}
                
                // response
                response.object = function () {
                    var queries = [],
                        results = [];
                    
                    queries[0] = "select * from yql.storage where name = '{select}'"
                        .supplant( { 'select' : select } );
                    results[0] = y.xmlToJson( y.query(queries[0]).results );
                    
                    if ( !results[0].results.result.value || !results[0].results.result.value[key] ) {
                        return {
                            error : 'key not found'
                        }
                    }
                    
                    results[0].results.result.value[key] = val;
                    
                    queries[1] = "update yql.storage set value='{value}' where name='{update}'"
                        .supplant( { 
                            'value' : JSON.stringify( results[0].results.result.value ), 
                            'update' : update
                        } );
                    results[1] = y.xmlToJson( y.query( queries[1] ).results );
                    
                    return results[1].results
                }();
            </execute>
        </update>
        <delete>
            <!-- sample query: use 'http://{your domain}/kv.xml' as kv; delete from kv where key='foo' -->
            <!-- note: use env file to define table url, select, & update -->
            <urls><url></url></urls>
            <inputs>
                <key id="select" type="xs:string" paramType="variable" required="true"/>
                <key id="update" type="xs:string" paramType="variable" required="true"/>
                <key id="key" type="xs:string" paramType="variable" required="true"/>
            </inputs>
            <execute>
                // http://www.JSON.org/json2.js w/ alert removed
                y.include('http://{your domain}/json2.js');
                
                // supplant fn (credit: http://www.crockford.com/javascript/remedial.html)
                if(!String.prototype.supplant){String.prototype.supplant=function(o){return this.replace(/{([^{}]*)}/g,function(a,b){var r=o[b];return typeof r==='string'||typeof r==='number'?r:a;});};}
                
                // response
                response.object = function () {
                    var queries = [],
                        results = [];
                    
                    queries[0] = "select * from yql.storage where name = '{select}'"
                        .supplant( { 'select' : select } );
                    results[0] = y.xmlToJson( y.query(queries[0]).results );
                    
                    if ( !results[0].results.result.value || !results[0].results.result.value[key] ) {
                        return {
                            error : 'key not found'
                        }
                    }
                    
                    delete results[0].results.result.value[key];
                    
                    queries[1] = "update yql.storage set value='{value}' where name='{update}'"
                        .supplant( { 
                            'value' : JSON.stringify( results[0].results.result.value ), 
                            'update' : update
                        } );
                    results[1] = y.xmlToJson( y.query( queries[1] ).results );
                    
                    return results[1].results
                }();
            </execute>
        </delete>
    </bindings>
</table>

Please post back w/ suggestions/questions.  Here’s a duck for good luck:

Flickr picture of a mallard

Photo credit: foxypar4

notes from Cloud Expo 2009: Surendra Reddy’s presentation on “Walking Through Cloud Serving at Yahoo!”

open cloud access protocol (opencap)
– definition, deployment, and life cycle mgmt of cloiud resources
– aloc, provisioning, and metering of clourd resources
– metasdat,/registry for cloud resources
– virtual infrastructure
– why ietf?  they are brilliant folks.  we’re not attaching any vendor-specific details
– open source implentation planned
– structure
— resource model
— all infastructure
— nodes, networks, etc
— resource properties
— modeled as json objects
— standard catalog of attributes, extensible
— resource operations
— operation, control, etc.
— management ntification services

– smtp is simple.  protocols must be simple
– traffic server has bindings built in (or vice versa?)
– open cloud consortium
— a national testbed to bring clouds together

Notes from Cloud Expo 2009: Raghu Ramakrishnan’s talk on the Yahoo! cloud: “key challenges in cloud computing … and the yahoo! approach”

raghu ramakrishnan
– a triumphant preso
– “key chalengeds in cloud comoputing .. and the y! approach”

this is a watershed time.  we’ve spent lots of time building packabged software now wer’re moving to the cloud

key challenges
– elastic scaling
– availabiolity
— if the cloud goes down, everyone is hosed.  consistency or performance myst be traded for availoability.
– handliong failures
— if things go wrong, what can the developer count on when things come up?
– operational efficiency
— cloud managers are db admins for 1000s of clients
– the right abstractions

yahoo’s cloud
– the cloud is an ecosystem.  it’s bigger than a single componenet.  all the pueces must work together seamlessly.

data management in the cloud
– how to make sense of the many options
– what are you trying todo?
– oltp vs olap
– oltp
— random access to a few records
— read-heavy vs write-heavy
– olap
— scan access to a large number of records
— by rows vs columns vs unstructired
– storage
— common features
— managed service. rest apis
— replication
— global footprint
— sherpa
— mopbstor

y! storage problem
– small records, 100kb or less
– structured records, lots of fields
– extreme data scale

typical applications
– user logins and profiles
— single=-record transactions suffice
– events
— alerts, social network activity
— ad clicks
app-specific data
– postings to messsage boards
– uploaded photos and tags

vsld data serving stores
– scale based on partitioning data accross machines
– range selections
— requests span machines
– availability
– replication
– durability
— is it required?
– how is data stored on a single machine?

the cap theorem
– consistency vs availability vs partition tolerance
– consistency => serializability

approaches to cap
– use a single version of a db w/ defered reconciliation
– defer transaction commit
– eventual consistency eg dynamo
– restrict transatctions eg sharded mysql
– object timelines, eg sherpa
– ref: julianbrowne.cim/artice/viewer/brewers-cap-theorem

single slide hadoop primer
– hadoop is wrte optimized, not ideal for serving

out there in the world
– oltp
— oracle, mysql,
— write optimized: cassandra
— main-mem; memchached

ways of using hadoop
– data workloads -> olap -> pig for row ops, zebra for column ops, map reduce for others

hadoop based apps
– we own the terasort benchmark

sherpa`
– parallel db
– geo replication
– structured, flexible schemas
– hashed and ordered tables
– components
— req -> routers -> (record looked up, if necessary) -> lookup cached -> individual machine
– raghu is awesome (“And then!”, sprinting through dense slides)
– write-ahead
– asynch replication
— why? we’re doing geo replication due to the physics involved
— supposing an eearthquake hits and ca falls in th ocean, two users can continue to update their profiles
– consistency model
— acid requiores synch updates
— eventual consistency works
— is there any middle ground?
— sherpa follows a timeline of changes achieved through a standard per-record primary copy protocol

operability
– cloud allows us to apperate at scale
– tablet splitting and balancing
– automatic transfer of mastership

comparing systems
– main point: all of this needs to be thought through and handled automatically

example
– sherpa, oracle, mysql work well for oltp

banchmark tiers
– cluster performance
– replication
– scale out
– availability
– we’d like to do this a group effort, in keeping w/ our philosophy

the integrated cloud
– big idea: declrative lang for specifying structure of service
– key insight: multi-env
– central mechanism: the integrated cloud
– surrendra will talk about htis

foundation componenets
– how to describe app
– desc for resources, entrypoijts, bindings, etc

yst hadled 16.4 million uniques for mj death news

acm socc
– acm symposium on cloud computing

notes from YUIConf 2009: “Building YUI 3 Custom Modules”, by Caridy Patino

what is a module in yui 3?
– modules are not plugins, but there is a plugin module
– module names are passed into a sandbox w/ the ‘use’ method
– prefer YUI().use instead of var Y = new YUI(); Y.use …
– you can have multiple use() calls to defer loading
– community modules vs basic yui core team modules

custom modules
– registration
— by seed YUI().use
— seed will import
— by inclusion
— manually add script include and then YUI.use
— YUI(config)
— most performant
— this takes advantage of onload handling
— reduces number of http requests req’d in ‘by inclusion’
— organization
—- use YUI_config global var to manage registration
—- you can have multiple config options

building custom modules
– YUI.add(‘foo’, fn(Y){mod code}, version, requirement list);
– naming convention: utilities are all lowercase, classes are camelcase w/ uppercase leading char
– plugins extend host modules
– stack: utilities –> classes –> plugins –> mashups

how to use and build plugins
– plugins allow us to extend an existing class at runtime
– the def of a plugin looks much like that for a module class
– instead of extending y.base, we extend y.plugin.base

mashups and legacy code
– using multiple modules, including external dependencies, enhancing dom, defining event listeners
– use case: using a pre-existing yui2-based object in yui3
– check out zakas’ talkon scalable app arch
– cool: organize app as module repo
– conclusions
— define apps at a granular level
— modular apps are easier to test
— share code thru yui3 gallery
— use yui custom modules to integrate pre-existing code

q/a
– differences btwn yui2 and yui3 lazy loading?
— yui3 will load everything as a single item, if module requirements are defined using config option
— yui3 will load items in the order they are specified
– reusing modules across multiple sandbox
— yes, if defined as such in config

github/caridy
twitter/caridy
caridy.name

notes: Chi-Hua Chien (Kleiner Perkins Caufiled & Byers) Keynote @ iphonedevcamp ’09

Chi-Hua Chien (Kleiner Perkins Caufiled & Byers) Keynote @ iphonedevcamp ‘09
Chi-Hua Chien (Kleiner Perkins Caufiled & Byers) Keynote @ iphonedevcamp ‘09

iphonedevcamp 09 friday keynote intro
– BT is the music guest
— did the Monster soundtrack.
— He’s a legend in the electronic scene.
— He has an app (“sonify”) in the appstore.  It performs a stutter edit when shaken

Chi-Hua Chien (Kleiner Perkins Caufiled & Byers) Keynote
– $100M ifund
– KPCB runs it
– # users of mobile internet is 4x desktop users, so it’s a huge opportunity
– how did the iphone change things?
— high speed internest
— consumer-oriented biz behind it
— appstore
— unified platform
– how is the fund doing?
— iphone sales dwarf ipod sales
— news, games, music usage on iphone far exceeds that on desktops
— 1.5B downloads from appstore in 165 days
— 4000 proposals reviewed
— 250 companies met
— actively engaged w/ 5 companes
— $45M deployed; will probably exceed $100M cap
— ifund companies: pelago, gogii, icontrol, ngmoco, booyah, mobshop, +1 unannounced
– ideas to consider
— intro
— #1 app in appstore (100k downloads/day): “do not push the red button”.  Does nothing!
— current top apps are a bit disappointing, non-innovative
— general thoughts
— innovation is key
— small, hacky apps are not long-term
— $0.06 ecp currently for ad space on apps
— categories
— mobile commerce
—- hypothesis: will be 3-4x size of commerce on internet ($100B), ie close to $1T
—- not too long ago, we were willing to pay $1 for info on 411
—- phones are great for spontaneous purchases
—- what will be the amazon, google of the mobile world
— realtime everywhere & anywhere
—- the internet was a really good newspaper; after you walk away from the computer, the internet is over.
—- the mobile internet knows a lot more about you
—- the mobile internet is always on
— local search
—- local search is completely different from desktop search, ie social, geo are bigger factors
— healthcare
—- mobile device can transmit monitoring data from embedded device
—- m. device can also analyze data
—- us gov is going to spend a lot to digitize med. data
— augmented reality
—- mobile device overlays data on top of real world
— real-world gaming
—- online games have spent a lot to build massive virtual worlds.  what if the planet was the world you played in?  what would ou play?
— QA
— size of funding & size of teams?
—- pelago $18M, 3 ppl
—- gogii $25M(?), 3
—- ngmoco $15M, 3 ppl
—- booyah $5M, 3 ppl
—- mobshop $5M, 1 guy, worked out of KPCB
—- icontrol 25-30 ppl
— min size of team?
—- 1 person
— can u form a company w/ 1 person?
—- yes, the laywers can work that out
— how can a vc firm, fund these companies, given the “capriciousness” of the apple review process?
—- KPCB has a very good relationship w/ apple
—- don’t try to launch a mobile operating sys.
— eductaional apps?
—- mobie is a componenet of ed
—- huge market
— top 5 things that matter when selecting a company to fund?
—- market, team (do you know what you’re doing), tech (is it defensible), prod (compelling?), biz. model (how are you going to make money)
— 4k proposals, $100M avail, are we on track?
—- ballpark, this is a really fast pace
—- KP has 3 ractices: life, info, green
—- 10k biz plans fund 15 comapnies/year
— signal to noise?
—- comperable to other biz areas
—- 300 in 1st 24/hrs
—- former web host has published early proposals
— level of detail in projections?
—- google’s orig. biz plan didn’t account for $1B in first years
—- no way to get it right
—- most important is how you think about it
— do you encourage fin. projections? yes
— preferred method of receiving proposals? email
— do you need a full biz plan?
—- 35 pages used to be the norm
—- now a deck that covers the 5 main areas is sufficient
— given current apps are disapponting, how could apple improve this?
—- apple selects apps that push the platform
—- fb app store was a freeforall
—- all the great apps we could have built around fb never happened because fb had no editorial
—- we’re going thru a market education process
—- the media hasn’t given enough notice to apps that are changing the world
— are these slides avail somewhere?
—- they’ll be on the ipdc website
— how important are barriers to entry?  how many have patents?
—- very imp., and almost all have patents
—- very difficult tech. is more imp. than patents
— hardware, software, accessories?
—- hardware takes a lot of capital, so it’s noty a vc-backed
//who’s bored?  ok, i’ll take 2 more
— to what extent has the pace of dev. in the iphone space affected KP biz. practice?
—- they impl. salesforce behind the site to hadnle the scale of submissions
—- this is still a very personal, high-touch biz
— I have been awarded a project (online sales to iphone), can we get funding?
—- no, KP is not investing in services
—- KP is looking for ppl developing IP
— what’s the dominant revenue model?
—- user-purchase, either virtual goods or real
—- thru apple’s payments platform
— are there tools deleopers can use to promote beyond the appstore, ie how to promote?
—- mobclix, adworld, tapjoy, etc run ad campaigns outside appstore
—- apple has sold 45M iphones/ipod touch, vs billions of handhelds sold overall
— make a lot from a few users, or a little from a lot?
—- depends on market: medical apps cost a lot; tons of free apps that are ad supported.
— would it help or hinder to have multiple disparate projects?
—- focus is best
—- pick the best one
— any segmentation among the funded projects?
—- not at the outset, but now we can see some in the data
—- gogii is loved by teens because their parents won’t get them a texting plan
— iterative dev?
—- yes, the app store looks more like internet dev, than packaged goods
— size of exit from 5M invested?
—- material, pub. companies, are coming.  this is the goal of KP
cchien@kpcb.com

notes: Bayjax Meetup @ Yahoo! Sunnyvale (7/27): Crockford “The JSON Saga”

Douglas Crockford describing The JSON Saga
Douglas Crockford talking about "The JSON Saga"

 

notes from Bayjax Meetup: http://www.meetup.com/BayJax/calendar/108524

Doug Crockford on the json saga

– json already existed in nature, but crockford discovered it

– it wwas being used at netscape in he form of array literals in communication 5 yrs before crockford discovered it

– crockford’s first transmission used js embedded in html in a frame for cross-browser compatability

– they set document.domain to get around the same origin policy

– backslashes are tolerated in json so we can put html in json

– original name was JSML, but that conflicted w/ a pre-existing java protcol

– json’s good for interserver and db applications

– some of his customers balked at usage because it wasn’t standard, so crockford put up a website to standardize it

– json is the intersection of modern programming languages

— simple values

— sequence of values

— collection of named values

— an intersection is easier to find than a union

– js is brilliant for state machines

– most ppl implement json parsers using eval, but this must be guarded by regex to valiudate the json, which slows it doen

– the latest version of ecma script implements a native JSON.parse, which is ver fast

– ajax was an important influence on json uptake

– improvements

— strip comments 

— comments broke the parser

— comment parsing greatly increased complexity

— alignment w/ yaml

— added e notation to numnbers

– no version number

— everything’s crap until 3.0, but we avoid this by not having version numbers

— perhaps someday it will be replaced

— at least one piece of the stack will remain constant forever

– minimalism

— can fit on the back of a business card

– influences

— lisp s-expressions

— perhaps the greatest influence

— rebol

— al built upon a represenation of data, which is then executable

— rebol is a brilliant language

— JS, Python, NewtonScript

— Brendon Ike is a brilliant guy, so it’s no accident that it has brilliant aspects

— all were developed in isolation at the same time

— NeXT

— OpenStep property lists

— XML

— how did it become so popular?

—- html reduced it to basics, made it more basic, and made it easier to get everything to balance

—- A-level CTOs threw it out, but the B- and C-level developers embraced it and outnumbered the A-levels

— john seely brown said “maybe only something this simple could work”

—- he thought that the future was in loosely coupled systems

— xmlsucks.org

— some guy named pault lists all the xml alternatives

– disruption

– the 1st rule of workmanship: use the right tool for the right job

– xml arose out of: one tool to rule them all

– where did the idea come from that every data format should be a document format?

— runoff was one of the first

— GML

— Scribe

— the first place where document formate were done well

— separated format from markup

— if the web had been based on scribe instead of sgml, it would be a better place today

— scribe was the first time where doc format was used for data

– license

— MIT + “the software shall be used for good, not evil”

– the logo

— related to the ambihelical hexnut 😛

— a square and a circle w/ a twist

— data interchange we can believe in

– questions

— what would make html better

— make it extensible

— to be able to define new tags using css

— is there case-sensitivity in unicode?

— maybe.  use lower-case in the meantime

— what would you like to see replace json

— jsonp is brilliant

— currently, we can’t easily represent simple bags

— remove the quotes from the keys

— a = []; a[0] = a; => infinite loop when run thru parser

— schema-less langs

— i don’t care

— brilliant work in schema-less dbs now

— why no commnets

— because ppl were using them to communicate w/ the parser

doug crockford on the json saga
– json already existed in nature, but crockford discovered it
– it wwas being used at netscape in he form of array literals in communication 5 yrs before crockford discovered it
– crockford’s first transmission used js embedded in html in a frame for cross-browser compatability
– they set document.domain to get around the same origin policy
– backslashes are tolerated in json so we can put html in json
– original name was JSML, but that conflicted w/ a pre-existing java protcol
– json’s good for interserver and db applications
– some of his customers balked at usage because it wasn’t standard, so crockford put up a website to standardize it
– json is the intersection of modern programming languages
— simple values
— sequence of values
— collection of named values
— an intersection is easier to find than a union
– js is brilliant for state machines
– most ppl implement json parsers using eval, but this must be guarded by regex to valiudate the json, which slows it doen
– the latest version of ecma script implements a native JSON.parse, which is ver fast
– ajax was an important influence on json uptake
– improvements
— strip comments 
— comments broke the parser
— comment parsing greatly increased complexity
— alignment w/ yaml
— added e notation to numnbers
– no version number
— everything’s crap until 3.0, but we avoid this by not having version numbers
— perhaps someday it will be replaced
— at least one piece of the stack will remain constant forever
– minimalism
— can fit on the back of a business card
– influences
— lisp s-expressions
— perhaps the greatest influence
— rebol
— al built upon a represenation of data, which is then executable
— rebol is a brilliant language
— JS, Python, NewtonScript
— Brendon Ike is a brilliant guy, so it’s no accident that it has brilliant aspects
— all were developed in isolation at the same time
— NeXT
— OpenStep property lists
— XML
— how did it become so popular?
—- html reduced it to basics, made it more basic, and made it easier to get everything to balance
—- A-level CTOs threw it out, but the B- and C-level developers embraced it and outnumbered the A-levels
— john seely brown said “maybe only something this simple could work”
—- he thought that the future was in loosely coupled systems
— xmlsucks.org
— some guy named pault lists all the xml alternatives
– disruption
– the 1st rule of workmanship: use the right tool for the right job
– xml arose out of: one tool to rule them all
– where did the idea come from that every data format should be a document format?
— runoff was one of the first
— GML
— Scribe
— the first place where document formate were done well
— separated format from markup
— if the web had been based on scribe instead of sgml, it would be a better place today
— scribe was the first time where doc format was used for data
– license
— MIT + “the software shall be used for good, not evil”
– the logo
— related to the ambihelical hexnut 😛
— a square and a circle w/ a twist
— data interchange we can believe in
– questions
— what would make html better
— make it extensible
— to be able to define new tags using css
— is there case-sensitivity in unicode?
— maybe.  use lower-case in the meantime
— what would you like to see replace json
— jsonp is brilliant
— currently, we can’t easily represent simple bags
— remove the quotes from the keys
— a = []; a[0] = a; => infinite loop when run thru parser
— schema-less langs
— i don’t care
— brilliant work in schema-less dbs now
— why no commnets
— because ppl were using them to communicate w/ the parserdoug crockford on the json saga
– json already existed in nature, but crockford discovered it
– it wwas being used at netscape in he form of array literals in communication 5 yrs before crockford discovered it
– crockford’s first transmission used js embedded in html in a frame for cross-browser compatability
– they set document.domain to get around the same origin policy
– backslashes are tolerated in json so we can put html in json
– original name was JSML, but that conflicted w/ a pre-existing java protcol
– json’s good for interserver and db applications
– some of his customers balked at usage because it wasn’t standard, so crockford put up a website to standardize it
– json is the intersection of modern programming languages
— simple values
— sequence of values
— collection of named values
— an intersection is easier to find than a union
– js is brilliant for state machines
– most ppl implement json parsers using eval, but this must be guarded by regex to valiudate the json, which slows it doen
– the latest version of ecma script implements a native JSON.parse, which is ver fast
– ajax was an important influence on json uptake
– improvements
— strip comments 
— comments broke the parser
— comment parsing greatly increased complexity
— alignment w/ yaml
— added e notation to numnbers
– no version number
— everything’s crap until 3.0, but we avoid this by not having version numbers
— perhaps someday it will be replaced
— at least one piece of the stack will remain constant forever
– minimalism
— can fit on the back of a business card
– influences
— lisp s-expressions
— perhaps the greatest influence
— rebol
— al built upon a represenation of data, which is then executable
— rebol is a brilliant language
— JS, Python, NewtonScript
— Brendon Ike is a brilliant guy, so it’s no accident that it has brilliant aspects
— all were developed in isolation at the same time
— NeXT
— OpenStep property lists
— XML
— how did it become so popular?
—- html reduced it to basics, made it more basic, and made it easier to get everything to balance
—- A-level CTOs threw it out, but the B- and C-level developers embraced it and outnumbered the A-levels
— john seely brown said “maybe only something this simple could work”
—- he thought that the future was in loosely coupled systems
— xmlsucks.org
— some guy named pault lists all the xml alternatives
– disruption
– the 1st rule of workmanship: use the right tool for the right job
– xml arose out of: one tool to rule them all
– where did the idea come from that every data format should be a document format?
— runoff was one of the first
— GML
— Scribe
— the first place where document formate were done well
— separated format from markup
— if the web had been based on scribe instead of sgml, it would be a better place today
— scribe was the first time where doc format was used for data
– license
— MIT + “the software shall be used for good, not evil”
– the logo
— related to the ambihelical hexnut 😛
— a square and a circle w/ a twist
— data interchange we can believe in
– questions
— what would make html better
— make it extensible
— to be able to define new tags using css
— is there case-sensitivity in unicode?
— maybe.  use lower-case in the meantime
— what would you like to see replace json
— jsonp is brilliant
— currently, we can’t easily represent simple bags
— remove the quotes from the keys
— a = []; a[0] = a; => infinite loop when run thru parser
— schema-less langs
— i don’t care
— brilliant work in schema-less dbs now
— why no commnets
— because ppl were using them to communicate w/ the parser

notes: Bayjax Meetup (7/27) @ Yahoo! Sunnyvale: Jon Leblanc on YQL + YUI

 

Jon Leblanc talking about YQL + YUI
Jon Leblanc talking about YQL + YUI

 

 

Meetup: http://www.meetup.com/BayJax/calendar/10852424/

jonleblanc on using yui w/ yql
– “select * from internet” gets a laugh
– the console (developer.yahoo.com/yql/console) is an easy way to get started w/ yql
– there are ~80 tables of y! data
– 253 community tables on github.com/yql
– yql execute
– yql insert|update|delete
– christian heilmann’s “geomaker” tool scrapes a url, extracts locations, and plots them on a map
– github.com/jsleblanc/yql-utilities
— js-yql-display
— yql can return jsonp and jsonpx
— yql plays well w/ yui
– questions
— rate limiting
— 1000/hr/ip w/ oauth
— w/ oauth 100k/day/ip
— caching?
— defaults to 5 min
— build your own table or append a var to url to break the cache

jonleblanc on using yui w/ yql

– the console (developer.yahoo.com/yql/console) is an easy way to get started w/ yql

– there are ~80 tables of y! data

– 253 community tables on github.com/yql

– yql execute

– yql insert|update|delete

– christian heilmann’s “geomaker” tool scrapes a url, extracts locations, and plots them on a map

– github.com/jcleblanc/yql-utilities

— js-yql-display

— yql can return jsonp and jsonpx

— yql plays well w/ yui

– links are on http://speakerrate.com/jcleblanc

– questions

— rate limiting

— 1000/hr/ip w/ oauth

— w/ oauth 100k/day/ip

— caching?

— defaults to 5 min

— build your own table or append a var to url to break the cache

notes: Bayjax Meetup @ Yahoo! Sunnyvale (7/27): Nicole Sullivan on OO CSS

 

Nicole Sullivan talking about Object Oriented CSS
Nicole Sullivan talking about Object Oriented CSS

 

 

meetup: http://www.meetup.com/BayJax/calendar/10852424/

nicole sullivan (@stubbornella) on object-oriented css

– slides are on slideshare/stubbornella

– how we are doing css wrong:

— we require expert-level developers to be effective

— filesize is growing out of control

— code re-use is almost nonexistant

— code is too fragile

– most important mistake: we write overly clever modules; everything is a one-off

— so size increases best-case at a 1-1 rate

– so what is oocss?

— stubbornella did write an open-source framework

— oocss lives to the left of the curly braces

– pieces: selctors

— the size of the css file is one of the largest factors in css performance; focus on http requests

— reflows and rendering is not that important

— duplication is worse than stale rules

— define default values; don’t repeat defaults

— define structure in a separate class

— style classes rather than elements; define styles to be dom-independant

— avoid styling elements; define styles in classes

— give all our rules the same strength; make every rule to have the same speceficity

— use hacks sparingly

— we should only need hacks for ie 5.5, 6, and maybe 7; nothing else requires it

— use underscore and star instead of js to apply browser hacks

— avoid specifying location, eg use .sidenave instead of .nav ul

— avoid overly-specific classes

— avoid singeltons, ie aoid using ids

— ids kill re-use

— use mixins

— use encapsulation

— if an object can live on its own, use wrapper classes.  Otherwise, avoid cascading

– heading

— componenets are like reusable legos

— > reusing elements makes them performance freebees <

— avoid duplication

— avoid nearly identical modules

— rule: if two modules are two similar to include next to each other, they’re too close for the same site

— avoid location-depemdent styles

— “HEADING” shouldn’t become “heading” on another part of the page

— define global defaults

— apply styles to classes instead of elements

—- respects semantics while allowing visual flexibility

— do we really need more than 6 headings?

– module

– grid

– questiojns

— if the html is broken, nice css wont work, right?

— yes.  a css obj is composed of html and css

— any research into compilation?

— the w3c should implement “extends” and “inherits” instead of us using compilation