|
Server : Apache/2.2.17 (Unix) mod_ssl/2.2.17 OpenSSL/0.9.8e-fips-rhel5 DAV/2 PHP/5.2.17 System : Linux localhost 2.6.18-419.el5 #1 SMP Fri Feb 24 22:47:42 UTC 2017 x86_64 User : nobody ( 99) PHP Version : 5.2.17 Disable Function : NONE Directory : /proc/21573/root/usr/lib/python2.4/site-packages/urlgrabber/ |
Upload File : |
mò
AÝrMc @ s˜ d k Z d k Z d k Z d k Z d k Z d Z y d k l Z Wn" e j
o Z
d k l Z n Xd e f d „ ƒ YZ d e i
f d „ ƒ YZ d e f d „ ƒ YZ d f d „ ƒ YZ d
e i f d „ ƒ YZ d k l Z l Z l Z l Z l Z l Z l Z d k Z d k Z d k Z d k Z d k Z d k Z d
e i f d „ ƒ YZ d e i! f d „ ƒ YZ! d a" d „ Z# d „ Z$ d „ Z% d S( N( s StringIOt
RangeErrorc B s t Z d Z RS( s6 Error raised when an unsatisfiable range is requested.( t __name__t
__module__t __doc__( ( ( t8 /usr/lib/python2.4/site-packages/urlgrabber/byterange.pyR # s t HTTPRangeHandlerc B s t Z d Z d „ Z d „ Z RS( sº Handler that enables HTTP Range headers.
This was extremely simple. The Range header is a HTTP feature to
begin with so all this class does is tell urllib2 that the
"206 Partial Content" reponse from the HTTP server is what we
expected.
Example:
import urllib2
import byterange
range_handler = range.HTTPRangeHandler()
opener = urllib2.build_opener(range_handler)
# install it
urllib2.install_opener(opener)
# create Request and set Range header
req = urllib2.Request('http://www.python.org/')
req.header['Range'] = 'bytes=30-50'
f = urllib2.urlopen(req)
c C s1 t i | | | i ƒ ƒ } | | _ | | _ | S( N( t urllibt
addinfourlt fpt hdrst reqt get_full_urlt rt codet msg( t selfR
R R
R R R ( ( R t http_error_206? s c C s t d ƒ ‚ d S( Ns Requested Range Not Satisfiable( R ( R R
R R
R R ( ( R t http_error_416F s ( R R R R R ( ( ( R R ' s t HTTPSRangeHandlerc B s t Z d Z d „ Z d „ Z RS( s! Range Header support for HTTPS. c C s | i | | | | | ƒ S( N( R R R
R R
R R ( R R
R R
R R ( ( R t https_error_206M s c C s | i | | | | | ƒ d S( N( R t https_error_416R
R R
R R ( R R
R R
R R ( ( R R P s ( R R R R R ( ( ( R R J s t RangeableFileObjectc B sh t Z d Z d „ Z d „ Z d „ Z d d „ Z d d „ Z d d „ Z d „ Z d
„ Z
d „ Z RS( s" File object wrapper to enable raw range handling.
This was implemented primarilary for handling range
specifications for file:// urls. This object effectively makes
a file object look like it consists only of a range of bytes in
the stream.
Examples:
# expose 10 bytes, starting at byte position 20, from
# /etc/aliases.
>>> fo = RangeableFileObject(file('/etc/passwd', 'r'), (20,30))
# seek seeks within the range (to position 23 in this case)
>>> fo.seek(3)
# tell tells where your at _within the range_ (position 3 in
# this case)
>>> fo.tell()
# read EOFs if an attempt is made to read past the last
# byte in the range. the following will return only 7 bytes.
>>> fo.read(30)
c C s> | | _ t | ƒ \ | _ | _ d | _ | i | i ƒ d S( s Create a RangeableFileObject.
fo -- a file like object. only the read() method need be
supported but supporting an optimized seek() is
preferable.
rangetup -- a (firstbyte,lastbyte) tuple specifying the range
to work over.
The file object provided is assumed to be at byte offset 0.
i N( t foR t range_tuple_normalizet rangetupt firstbytet lastbytet realpost _do_seek( R R R ( ( R t __init__h s
c C s4 t | i | ƒ o t | i | ƒ Sn t | ‚ d S( s« This effectively allows us to wrap at the instance level.
Any attribute not found in _this_ object will be searched for
in self.fo. This includes methods.N( t hasattrR R t namet getattrt AttributeError( R R ( ( R t __getattr__v s c C s | i | i S( s5 Return the position within the range.
This is different from fo.seek in that position 0 is the
first byte position of the range tuple. For example, if
this object was created with a range tuple of (500,899),
tell() will return 0 when at byte position 500 of the file.
N( R R R ( R ( ( R t tell~ s i c C s˜ | d j o | i | } n< | d j o | i | } n | d j o t d ƒ ‚ n | i o | | i j o
| i } n | i | | i ƒ d S( se Seek within the byte range.
Positioning is identical to that described under tell().
i i i s$ seek from end of file not supported.N( t whenceR R t offsett
realoffsetR t IOErrorR R ( R R% R$ R& ( ( R t seek‡ s
iÿÿÿÿc C s: | i | ƒ } | i i | ƒ } | i t | ƒ 7_ | S( s` Read within the range.
This method will limit the size read based on the range.
N( R t _calc_read_sizet sizeR t readt rsltR t len( R R* R, ( ( R R+ š s
c C s: | i | ƒ } | i i | ƒ } | i t | ƒ 7_ | S( sf Read lines within the range.
This method will limit the size read based on the range.
N( R R) R* R t readlineR, R R- ( R R* R, ( ( R R. £ s
c C s^ | i oP | d j o/ | i | | i j o | i | i } qV qZ | i | i } n | S( sS Handles calculating the amount of data to read based on
the range.
iÿÿÿÿN( R R R* R ( R R* ( ( R R) ¬ s
c C sN t | i d ƒ p | i | ƒ n | i i | i | ƒ | i | 7_ d S( s Seek based on whether wrapped object supports seek().
offset is relative to the current position (self.realpos).
R( N( R R R t _poor_mans_seekR% R( R ( R R% ( ( R R ¸ s c C sƒ d } d } xp | | j ob | | | j o | | } n | i i | ƒ } t | ƒ | j o t d ƒ ‚ n | | 7} q Wd S( s¯ Seek by calling the wrapped file objects read() method.
This is used for file like objects that do not have native
seek support. The wrapped objects read() method is called
to manually seek to the desired position.
offset -- read this number of bytes from the wrapped
file object.
raise RangeError if we encounter EOF before reaching the
specified offset.
i i s Requested Range Not SatisfiableN( t post bufsizeR% R R R+ t bufR- R ( R R% R0 R1 R2 ( ( R R/ Ã s
( R R R R R" R# R( R+ R. R) R R/ ( ( ( R R S s t FileRangeHandlerc B s t Z d Z d „ Z RS( s~ FileHandler subclass that adds Range support.
This class handles Range headers exactly like an HTTP
server would.
c C sÈ d k } d k } | i ƒ } | i ƒ } t i | ƒ }
t
i |
ƒ } | t i
} t i | t i ƒ } | i | ƒ d } | oO t i | ƒ \ } } | p t i | ƒ | i ƒ j o t i d ƒ ‚ qÔ n t |
d ƒ } | i i d d ƒ } t# | ƒ } | o} | \ } }
|
d j o
| }
n | d j p | | j p
|
| j o t& d ƒ ‚ n |
| } t' | | |
f ƒ } n | i( t) d | p d | | f ƒ ƒ } t i* | | d | ƒ S(
Ni s file not on local hostt rbt Ranget s Requested Range Not Satisfiables6 Content-Type: %s
Content-Length: %d
Last-modified: %s
s
text/plains file:(+ t mimetypest mimetoolsR
t get_hostt hostt get_selectort fileR t url2pathnamet localfilet ost statt statst ST_SIZER* t rfc822t
formatdatet ST_MTIMEt modifiedt
guess_typet mtypet splitportt portt sockett
gethostbynameR t get_namest urllib2t URLErrort openR t headerst gett Nonet branget range_header_to_tuplet fbt lbR R t Messaget StringIOR ( R R
R8 RT RH R< RJ R* RA RV R> R7 R: RW R RF RQ ( ( R t open_local_fileÜ s6
#
'
)( R R R RZ ( ( ( R R3 × s ( s splitports splitusers splitpasswds splitattrs unquotes addclosehooks
addinfourlt FTPRangeHandlerc B s t Z d „ Z d „ Z RS( Nc C sÆ | i ƒ } | p
t d ‚ n t | ƒ \ } } | d j o
t i } n
t | ƒ } t
| ƒ \ } } | o t | ƒ \ } }
n d }
t | ƒ } t | p d ƒ } t |
p d ƒ }
y t i | ƒ } Wn' t i j
o } t i | ƒ ‚ n Xt | i ƒ ƒ \ } } | i d ƒ } t t | ƒ } | d | d } } | o | d o | d } n y#| i | |
| | | ƒ } | o d p d }
xP | D]H } t | ƒ \ } } | i# ƒ d
j o | d j o | i$ ƒ }
q¡q¡Wd } t& | i' i( d d ƒ ƒ } | o' | \ } } | d j o
| } q<n | i, | |
| ƒ \ } } | o— | \ } } | d j o[ | d j p
| d j o t/ d ƒ ‚ n | } | | } | d j o t/ d ƒ ‚ qñqõ| | } t0 | d | f ƒ } n d } t1 i2 | i3 ƒ ƒ d } | o | d | 7} n | d j o | d j o | d | 7} n t5 | ƒ } t7 i8 | ƒ } t9 | | | i3 ƒ ƒ SWn4 t i: j
o% } t d | f t; i< ƒ d ‚ n Xd S( Ns ftp errors
no host givenR6 t /iÿÿÿÿi i t It Dt typet at At it dR5 s@ Requested Range Not Satisfiable due to unobtainable file length.s Requested Range Not Satisfiables Content-Type: %s
s Content-Length: %d
i ( s ftp errors
no host given( R` Ra Rb R] Rc R^ (= R
R9 R: R' RI RJ RS t ftplibt FTP_PORTt intt splitusert usert splitpasswdt passwdt unquoteRK RL t errorR RN RO t splitattrR; t patht attrst splitt dirst mapR< R t connect_ftpt fwR_ t attrt valuet lowert uppert restRU RQ RR t range_tupRV RW t retrfileR t retrlenR R R7 RG R RH RY t sfR8 RX R t
all_errorst syst exc_info( R R
R Ry RH Ro R< RJ RW Rz R_ Rq Rt Rj R R: Rh Rn R| Ru RV Rv RQ R} ( ( R t ftp_open s|
c C s t | | | | | ƒ } | S( N( t
ftpwrapperRh Rj R: RJ Rq Rt ( R Rh Rj R: RJ Rq Rt ( ( R Rs Z s ( R R R Rs ( ( ( R R[ s NR‚ c B s t Z d d „ Z RS( Nc
C s1 | i ƒ | d j o d } d } n d | } d } y | i i | ƒ Wn0 t i j
o! | i ƒ | i i | ƒ n Xd } | o.| o&y | i i
| ƒ Wn4 t i j
o% } t d | f t i ƒ d ‚ n X| i i | ƒ y# d | } | i i | | ƒ } Wq¹t i j
o’ } t | ƒ d
d j o; | i | | ƒ \ } } t | | d f ƒ } | | f Sqµt | ƒ d
d
j o t d | f t i ƒ d ‚ qµq¹Xn | pA | i i d ƒ | o d | } n d } | i i | ƒ } n d | _ t | d i d ƒ | i ƒ | d f S( NRc R^ s TYPE Ai s TYPE i s ftp errori s RETR i t 501R6 t 550s LIST t LISTR4 ( Rc R^ ( R t endtransferR_ t cmdt isdirt ftpt voidcmdRd R~ t initRS t connR< t nlstt
error_permt reasonR' R R€ t ntransfercmdRy t strR{ R R| R t busyt addclosehookt makefile(
R R< R_ Ry Rˆ R‡ R R R| RŒ ( ( R R{ c sF
!
) ( R R RS R{ ( ( ( R R‚ ^ s c C s¦ | d j o d Sn t d j o d k } | i d ƒ a n t i | ƒ } | oN t | i d d ƒ ƒ } | o' | d o | d | d d f } n | Sn f S( sØ Get a (firstbyte,lastbyte) tuple from a Range header value.
Range headers have the form "bytes=<firstbyte>-<lastbyte>". This
function pulls the firstbyte and lastbyte values and returns
a (firstbyte,lastbyte) tuple. If lastbyte is not specified in
the header value, it is returned as an empty string in the
tuple.
Return None if range_header is None
Return () if range_header does not conform to the range spec
pattern.
Ns ^bytes=(\d{1,})-(\d*)i i i ( t range_headerRS t _rangeret ret compilet matchR t groupt tup( R• R› R— R™ ( ( R RU “ s
c C s_ | d j o d Sn t | ƒ } | o3 | d o | d | d d f } n d | Sn d S( s• Convert a range tuple to a Range header value.
Return a string of the form "bytes=<firstbyte>-<lastbyte>" or None
if no range is needed.
i i s bytes=%s-%sN( Rz RS R ( Rz ( ( R t range_tuple_to_header® s
c C sì | d j o d Sn | d } | d j o
d } n
t | ƒ } y | d } Wn t j
o
d } n6 X| d j o
d } n | d j o t | ƒ } n | | f d j o d Sn | | j o t d | | f ƒ ‚ n | | f S( s7 Normalize a (first_byte,last_byte) range tuple.
Return a tuple whose first element is guaranteed to be an int
and whose second element will be '' (meaning: the last byte) or
an int. Finally, return None if the normalized tuple == (0,'')
as that is equivelant to retrieving the entire file.
i R6 i s Invalid byte range: %s-%sN( NR6 ( i R6 ( Rz RS RV Rf RW t
IndexErrorR ( Rz RV RW ( ( R R º s(
(& R? R@ R RN RC RS t DEBUGt cStringIORY t ImportErrorR R' R t BaseHandlerR R R t FileHandlerR3 RI Rg Ri Rm Rk R“ R Rd RK R R7 R8 t
FTPHandlerR[ R‚ R– RU Rœ R ( Rk R8 R Rœ R RI Rg R R R3 Rm R7 Rd R@ R“ RC R RN R R Rž R‚ R RK RY RU R Ri R[ R? ( ( R t ? s6 # „+1 S4