A jquery-like library for python

pyquery: a jquery-like library for python

pyquery allows you to make jquery queries on xml documents. The API is as much as possible the similar to jquery. pyquery uses lxml for fast xml and html manipulation.

This is not (or at least not yet) a library to produce or interact with javascript code. I just liked the jquery API and I missed it in python so I told myself "Hey let's make jquery in python". This is the result.

The project is being actively developped on a git repository on Github. I have the policy of giving push access to anyone who wants it and then to review what they do. So if you want to contribute just email me.

Please report bugs on the github issue tracker.

I've spent hours maintaining this software, with love. Please consider tiping if you like it:

BTC: 1PruQAwByDndFZ7vTeJhyWefAghaZx9RZg

ETH: 0xb6418036d8E06c60C4D91c17d72Df6e1e5b15CE6

LTC: LY6CdZcDbxnBX9GFBJ45TqVj8NykBBqsmT

Quickstart

You can use the PyQuery class to load an xml document from a string, a lxml document, from a file or from an url:

>>> from pyquery import PyQuery as pq
>>> from lxml import etree
>>> import urllib
>>> d = pq("<html></html>")
>>> d = pq(etree.fromstring("<html></html>"))
>>> d = pq(url=your_url)
>>> d = pq(url=your_url,
...        opener=lambda url, **kw: urlopen(url).read())
>>> d = pq(filename=path_to_html_file)

Now d is like the $ in jquery:

>>> d("#hello")
[<p#hello.hello>]
>>> p = d("#hello")
>>> print(p.html())
Hello world !
>>> p.html("you know <a href='http://python.org/'>Python</a> rocks")
[<p#hello.hello>]
>>> print(p.html())
you know <a href="http://python.org/">Python</a> rocks
>>> print(p.text())
you know Python rocks

You can use some of the pseudo classes that are available in jQuery but that are not standard in css such as :first :last :even :odd :eq :lt :gt :checked :selected :file:

>>> d('p:first')
[<p#hello.hello>]

python: 2.7.3 pyquery: master cssselect: 0.8.0

from pyquery import PyQuery as pq
doc = pq([])
doc._css_to_xpath(u'a:contains("我")') 
# => u"descendant-or-self::a[contains(text(), '\u6211')]"

doc._css_to_xpath(u'a:contains("我") span')

Traceback (most recent call last):
  File "test.py", line 7, in <module>
    doc._css_to_xpath(u'a:contains("我") span')
  File "...pyquery/pyquery.py", line 227, in _css_to_xpath
    return self._translator.css_to_xpath(selector, prefix)
  File "...cssselect-0.8-py2.7.egg/cssselect/xpath.py", line 188, in css_to_xpath
    for selector in selectors)
  File "...cssselect-0.8-py2.7.egg/cssselect/xpath.py", line 188, in <genexpr>
    for selector in selectors)
  File "...cssselect-0.8-py2.7.egg/cssselect/xpath.py", line 210, in selector_to_xpath
    return (prefix or '') + _unicode(xpath)
  File "...pyquery/cssselectpatch.py", line 35, in __unicode__
    return str(self).decode('utf-8')
  File "...pyquery/cssselectpatch.py", line 29, in __str__
    path = str(path)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u6211' in position 20: ordinal not in range(128)

After some discussion in this Stack Overflow question we realized that pyquery's text method doesn't match how html usually works. PyQuery inserts spaces in places where you wouldn't expect them.

For example:

>>> pq('<div><span>foo</span><span>bar</span></div>').text()
'foo bar'

But this renders as "foobar" in most (all?) browsers and jQuery's text() (which is just importing Sizzle's /getText()) would also return "foobar". Also see Section 16.6.1 of the CSS2.1 spec for how browsers are supposed to render whitespace.

I started writing a fix but I realized that this would break a lot of the tests as well as backwards-compatibility, so I'm submitting this issue instead!

I have renamed the camelCase methods to comply with the lower_case naming conventions endorsed by the PEP 8. For the sake of backward compatibility, I have left aliases to the camelCase methods.

I ran into a problem loading files with PyQuery on Python 3.3 :

$ python
Python 3.3.0 (default, Oct 22 2012, 11:17:43) 
>>> from pyquery import PyQuery as pq
>>> pq(filename="index.html")
[None]
>>> from lxml.html import parse
>>> parse("index.html")
<lxml.etree._ElementTree object at 0x108a72f38>
>>> pq(parse("index.html").getroot())
[<html>]

Is filename expected to work? I noticed that it's listed on http://pythonhosted.org/pyquery/ in the quickstart but not in the full API listing.

Hi,

Since Chrome upgraded, I have issues in the following code:

driver.get(url_) # Selenium part

jQuery = pq(self.driver.page_source) # PyQuery part

for zone in jQuery("table"): table = jQuery(zone) if table.attr("class") == "gf-table historical_price": ... here I read the content ...

Unfortunately, it worked perfectly before Google Chrome upgraded 2 or 3 months ago, and now I read no content at all. I have content in the jQuery variable but it seems impossible to read it as html in the loop.

As a workaround, I downgraded Chrome but for some reason, Chrome automatically upgrade while I blocked it. Anyway, that is a dirty solution.

Do you see where is the reason for this problem?

Regards, SD

A jquery-like library for python

Related tags

Overview

pyquery: a jquery-like library for python

Quickstart

Comments

UnicodeEncodeError

text() method adds whitespace

PEP 8, general cleanup

Open by filename fails on Python 3

Google Chrome+Selenium+PyQuery issue