pytool: Python helper package

This package contains a lot of helpful little methods and functions to make your life a little easier and your day a little better. Enjoy!

pytool.cmd: Command helpers

This module contains helpers related to writing scripts and creating command line utilities.

Command helpers

pytool.cmd.Command

class pytool.cmd.Command

Base class for creating commands that can be run easily as scripts. This class is designed to be used with the console_scripts entry point to create Python-based commands for your packages.

Hello world example:

# hello.py
from pytool.cmd import Command

class HelloWorld(Command):
    def run(self):
        print "Hello World."

The only thing that must be defined in the subclass is the run() method, which should contain the code to launch your application, all other methods are optional.

Example setup.py:

# setup.py
from setuptools import setup

setup(
    # ...
    entry_points={
        'console_scripts':[
            'helloworld = hello:HelloWorld.console_script',
            ],
        },
    )

When using an entry point script, the Command has a special console_script() method for launching the application.

Starting without an entry point script:

# hello.py [cont'd]
if __name__ == '__main__':
    import sys
    HelloWorld().start(sys.argv[1:])

The start() method always requires an argument - even if it’s just an empty list.

More complex example:

from pytool.cmd import Command

class HelloAll(Command):
    def set_opts(self):
        self.opt('--world', default='World', help="use a different "
                "world")
        self.opt('--verbose', '-v', action='store_true', help="use "
                "more verbose output")

    def run(self):
        print "Hello", self.args.world

        if self.args.verbose:
            print "Hola", self.args.world

Whenever there are arguments for a command, they’re made available for your use as self.args. This object is created by argparse so refer to that documentation for more information.

classmethod console_script()

Method used to start the command when launched from a distutils console script.

describe(description)

Describe the command in more detail. This will be displayed in addition to the argument help.

This automatically strips leading indentation but does not strip all formatting like the ArgumentParser(description='') keyword.

Example:

class MyCommand(Command):
    def set_opts(self):
        self.describe("""
            This is an example command. To use the example command,
            run it.""")

    def run(self):
        pass
opt(*args, **kwargs)

Add an option to this command. This takes the same arguments as ArgumentParser.add_argument().

reload()

Reloads the command.

run()

Subclasses should override this method to start the command process. In other words, this is where the magic happens.

set_opts()

Subclasses should override this method to configure the command line arguments and options.

Example:

class MyCommand(Command):
    def set_opts(self):
        self.opt('--verbose', '-v', action='store_true',
                help="be more verbose")

    def run(self):
        if self.args.verbose:
            print "I'm verbose."
start(args)

Starts a command and registers single handlers.

stop()

Exits the currently running process.

pytool.json: JSON helpers

This module contains helpers for working with JSON data.

Tries to use the simplejson module if it exists, otherwise falls back to the json module.

If the bson module exists, it allows bson.ObjectId objects to be decoded into JSON automatically.

JSON helpers

as_json()

pytool.json.as_json(obj, **kwargs)

Returns an object JSON encoded properly.

This method allows you to implement a hook method for_json() on your objects if you want to allow arbitrary objects to be encoded to JSON. A for_json() hook must return a basic JSON type (dict, list, int, float, string, unicode, float or None), or a basic JSON type which contains other objects which implement the for_json() hook.

If an object implements both _asdict() and for_json() the latter is given preference.

Also adds additional encoders for datetime and bson.ObjectId.

Parameters:
  • obj (object) – An object to encode.
  • **kwargs – Any optional keyword arguments to pass to the JSONEncoder
Returns:

JSON encoded version of obj.

New in version 2.4: Objects which have an _asdict() method will have that method called as part of encoding to JSON, even when not using simplejson.

New in version 2.4: Objects which have a for_json() method will have that method called and the return value used for encoding instead.

Changed in version 3.0: simplejson (>= 3.2.0) is now required, and relied upon for the _asdict() and for_json() hooks. This change may break backwards compatibility in any code that uses these hooks.

from_json()

pytool.json.from_json(value)

Decodes a JSON string into an object.

Parameters:value (str) – String to decode
Returns:Decoded JSON object

pytool.lang: Language helpers

This module contains items that are “missing” from the Python standard library, that do miscelleneous things.

Namespace

class pytool.lang.Namespace

Namespace object used for creating arbitrary data spaces. This can be used to create nested namespace objects. It can represent itself as a dictionary of dot notation keys.

Basic usage:

>>> from pytool.lang import Namespace
>>> # Namespaces automatically nest
>>> myns = Namespace()
>>> myns.hello = 'world'
>>> myns.example.value = True
>>> # Namespaces can be converted to dictionaries
>>> myns.as_dict()
{'hello': 'world', 'example.value': True}
>>> # Namespaces have container syntax
>>> 'hello' in myns
True
>>> 'example.value' in myns
True
>>> 'example.banana' in myns
False
>>> 'example' in myns
True
>>> # Namespaces are iterable
>>> for name, value in myns:
...     print name, value
...     
hello world
example.value True
>>> # Namespaces that are empty evaluate as False
>>> bool(Namespace())
False
>>> bool(myns.notset)
False
>>> bool(myns)
True
>>> # Namespaces allow the __get__ portion of the descriptor protocol
>>> # to work on instances (normally they would not)
>>> class MyDescriptor(object):
...     def __get__(self, instance, owner):
...         return 'Hello World'
...     
>>> myns.descriptor = MyDescriptor()
>>> myns.descriptor
'Hello World'

Namespaces are useful!

as_dict(base_name=None)

Return the current namespace as a dictionary.

Parameters:base_name (str) – Base namespace (optional)
iteritems(base_name=None)

Return iterator which returns (key, value) tuples.

Parameters:base_name (str) – Base namespace (optional)

UNSET

class pytool.lang.UNSET

Special class that evaluates to bool(False), but can be distinctly identified as seperate from None or False. This class can and should be used without instantiation.

>>> from pytool.lang import UNSET
>>> # Evaluates to False
>>> bool(UNSET)
False
>>> # Is a class-singleton (cannot become an instance)
>>> UNSET() is UNSET
True
>>> # Is good for checking default values
>>> if {}.get('example', UNSET) is UNSET:
...     print "Key is missing."
...     
Key is missing.
>>> # Has no length
>>> len(UNSET)
0
>>> # Is iterable, but has no iterations
>>> list(UNSET)
[]
>>> # It has a repr() equal to itself
>>> UNSET
UNSET

classproperty()

pytool.lang.classproperty(func)

Makes a @classmethod style property (since @property only works on instances).

from pytool.lang import classproperty

class MyClass(object):
    _attr = 'Hello World'

    @classproperty
    def attr(cls):
        return cls._attr

MyClass.attr # 'Hello World'
MyClass().attr # Still 'Hello World'

get_name()

pytool.lang.get_name(frame)

Gets the name of the passed frame.

Warning:It’s very important to delete a stack frame after you’re done using it, as it can cause circular references that prevents garbage collection.
Parameters:frame – Stack frame to inspect.
Returns:Name of the frame in the form module.class.method.

hashed_singleton()

pytool.lang.hashed_singleton(klass)

Wraps a class to create a hashed singleton version of it. A hashed singleton is like a singleton in that there will be only a single instance of the class for each call signature.

The singleton is kept as a weak reference, so if your program ceases to reference the hashed singleton, you may get a new instance if the Python interpreter has garbage collected your original instance.

This will not work for classes that take arguments that are unhashable (e.g. dicts, sets).

New in version 2.1.

Parameters:klass – Class to decorate

Example usage:

# Make a class directly behave as a hashed singleton
@hashed_singleton
class Test(object):
    def __init__(self, *args, **kwargs):
        pass

# Make an imported class behave as a hashed singleton
Test = hashed_singleton(Test)

# The same arguments give you the same class instance back
test = Test('a', k='k')
test is Test('a', k='k') # True

# A different argument signature will give you a new instance
test is Test('b', k='k') # False
test is Test('a', k='j') # False

# Removing all references to a hashed singleton instance will allow
# it to be garbage collected like normal, because it's only kept
# as a weak reference
del test
test = Test('a', k='k') # If the Python interpreter has garbage
                        # collected, you will get a new instance

singleton()

pytool.lang.singleton(klass)

Wraps a class to create a singleton version of it.

Parameters:klass – Class to decorate

Example usage:

# Make a class directly behave as a singleton
@singleton
class Test(object):
    pass

# Make an imported class behave as a singleton
Test = singleton(Test)

pytool.text: Text helpers

This module contains text related things that make life easier.

Text helpers

wrap()

pytool.text.wrap(text, width=70, indent='')

Return text wrapped to width while trimming leading indentation and preserving paragraphs.

This function is handy for turning indented inline strings into unindented strings that preserve paragraphs, whitespace, and any indentation beyond the baseline.

Parameters:
  • text (str) – Text to wrap
  • width (int) – Width to wrap text at (default: 70)
  • indent (str) – String to indent text with (default: ‘’)
>>> import pytool
>>> text = '''
        All this is indented by 8, but will be 0.
                This is indented by 16, and a really long long long
                line which is hard wrapped at a random character width,
                but will be wrapped appropriately at 70 chars
                afterwards.

        This is indented by 8 again.
    '''
>>> print pytool.text.wrap(text)
All this is indented by 8, but will be 0.
        This is indented by 16, and a really long long long line which
        is hard wrapped at a random character width, but will be
        wrapped appropriately at 70 chars afterwards.

This is indented by 8 again.
>>>

pytool.time: Time and Date related helpers

This module contains time related things that make life easier.

Timer

class pytool.time.Timer

This is a simple timer class.

timer = pytool.time.Timer()
for i in (1, 2, 3):
    sleep(i)
    print timer.mark(), "elapsed since last mark or start"
print timer.elapsed, "total elapsed"
elapsed

Return a timedelta of the time elapsed since the start.

mark()

Return a timedelta of the time elapsed since the last mark or start.

UTC

class pytool.time.UTC

UTC timezone. This is necessary since Python doesn’t include any explicit timezone objects in the standard library. This can be used to create timezone-aware datetime objects, which are a pain to work with, but a necessary evil sometimes.

from datetime import datetime
from pytool.time import UTC

utc_now = datetime.now(UTC())

as_utc()

pytool.time.as_utc(stamp)

Converts any datetime (naive or aware) to UTC time.

Parameters:stamp (datetime) – Datetime to convert
Returns:stamp as UTC time
from datetime import datetime
from pytool.time import as_utc

utc_datetime = as_utc(datetime.now())

utcnow()

pytool.time.utcnow()

Return the current UTC time as a timezone-aware datetime.

Returns:The current UTC time

fromutctimestamp()

pytool.time.fromutctimestamp(stamp)

Return a timezone-aware datetime object from a UTC unix timestamp.

Parameters:stamp (float) – Unix timestamp in UTC
Returns:UTC datetime object
import time
from pytool.time import fromutctimestamp

utc_datetime = fromutctimestamp(time.time())

toutctimestamp()

pytool.time.toutctimestamp(stamp)

Converts a naive datetime object to a UTC unix timestamp. This has an advantage over time.mktime in that it preserves the decimal portion of the timestamp when converting.

Parameters:stamp (datetime) – Datetime to convert
Returns:Unix timestamp as a float
from datetime import datetime
from pytool.time import toutctimestamp

utc_stamp = toutctimestamp(datetime.now())

is_dst()

pytool.time.is_dst(stamp)

Return True if stamp is daylight savings.

Parameters:stamp (datetime) – Datetime
Returns:True if stamp is daylight savings, otherwise False.

trim_time()

pytool.time.trim_time(stamp)

Trims the time portion off of stamp, leaving the date intact. Returns a datetime of the same date, set to 00:00:00 hours. Preserves timezone information.

Parameters:stamp (datetime) – Timestamp to trim
Returns:Trimmed timestamp

floor_day()

pytool.time.floor_day(stamp=None)

Return stamp floored to the current day. If no stamp is specified, the current time is used. This is similar to the date() method, but returns a datetime object, instead of a date object.

This is the same as trim_time().

Changed in version 2.0: Preserves timezone information if it exists, and uses pytool.time.utcnow() instead of datetime.datetime.now() if stamp is not given.

Parameters:stamp (datetime) – datetime object to floor (default: now)
Returns:Datetime floored to the day

floor_minute()

pytool.time.floor_minute(stamp=None)

Return stamp floored to the current minute. If no stamp is specified, the current time is used. Preserves timezone information.

New in version 2.0.

Parameters:stamp (datetime) – datetime object to floor (default: now)
Returns:Datetime floored to the minute

floor_month()

pytool.time.floor_month(stamp=None)

Return stamp floored to the current month. If no stamp is specified, the current time is used.

Changed in version 2.0: Preserves timezone information if it exists, and uses pytool.time.utcnow() instead of datetime.datetime.now() if stamp is not given.

Parameters:stamp (datetime) – datetime object to floor (default: now)
Returns:Datetime floored to the month

floor_week()

pytool.time.floor_week(stamp=None)

Return stamp floored to the current week, at 00:00 Monday. If no stamp is specified, the current time is used. Preserves timezone information.

This is the same as week_start()

New in version 2.0.

Parameters:stamp (datetime) – datetime object to floor (default now:)
Returns:Datetime floored to the week

make_week_seconds()

pytool.time.make_week_seconds(day, hour, minute=0, seconds=0)

Return week_seconds() for the given day of the week, hour and minute.

Parameters:
  • day (int) – Zero-indexed day of the week
  • hour (int) – Zero-indexed 24-hour
  • minute (int) – Minute (default: 0)
  • seconds (int) – Seconds (default: 0)
Returns:

Seconds since 00:00 Monday

week_seconds()

pytool.time.week_seconds(stamp)

Return stamp converted to seconds since 00:00 Monday.

Parameters:stamp (datetime) – Timestamp to convert
Returns:Seconds since 00:00 monday

week_seconds_to_datetime()

pytool.time.week_seconds_to_datetime(seconds)

Return the datetime that is seconds from the start of this week.

Parameters:seconds (int) – Seconds
Returns:Datetime for 00:00 Monday plus seconds

week_start()

pytool.time.week_start(stamp)

Return the start of the week containing stamp.

Changed in version 2.0: Preserves timezone information.

Parameters:stamp (datetime) – Timestamp
Returns:A datetime for 00:00 Monday of the given week

ago()

pytool.time.ago(stamp=None, **kwargs)

Return the current time as UTC minutes the specified timeframe.

This is a helper for simplifying the common pattern of pytool.time.utcnow() - datetime.timedelta(mintues=15).

Parameters:
  • stamp – An optional timestamp instead of utcnow()
  • days – Days previous
  • hours – Hours previous
  • minutes – Minutes previous
  • seconds – Days previous
Returns:

UTC timestamp

If you like brevity in your arguments, you can use the shorter versions, hrs=, mins= and secs=.

Or if you really want a short signature, you can use d=, h=, m=, s=.

import pytool

yesterday = pytool.time.ago(days=1)

a_little_while_ago = pytool.time.ago(minutes=1)

a_little_before = pytool.time.ago(my_date, hours=1)

# Shorter arguments
shorter = pytool.time.ago(hrs=1, mins=1, secs=1)

# Shorthand argument names
short = pytool.time.ago(d=1, h=1, m=1, s=1)

pytool.proxy: DictProxy and ListProxy classes

This module contains implmentations of proxy-list and proxy-dictionary objects.

DictProxy

class pytool.proxy.DictProxy(data)

Proxies all methods for a dict instance.

This is useful when you want to modify a dictionary’s behavior through subclassing without copying the dictionary or if you want to be able to modify the original dictionary.

Parameters:data – A dict or dict-like object (implements all the collections.MutableMapping methods)

New in version 2.2.

Note:If you intend to use a subclass which modifies the apparent keys or values of this class with pytool.json.as_json(), remember to override for_json() to produce the data you desire.

Example:

from pytool.proxy import DictProxy

class SquaredDict(DictProxy):
    def __getitem__(self, key):
        value = super(SquaredDict, self).__getitem__(key)
        if isinstance(value, int):
            value *= value
        return value

my_dict = {}
my_proxy = SquaredDict(my_dict)
my_proxy['val'] = 5

my_proxy['val'] # 25
my_dict['val'] # 5

ListProxy

class pytool.proxy.ListProxy(data)

Proxies all methods for a list instance. This is useful when you want to modify a list’s behavior without copying the list.

Methods which do not mutate a list, and instead return a new list will return a list instance rather than a ListProxy instance.

Parameters:data – A list or list-like object (implements all the collections.MutableSequence methods)

New in version 2.2.

Note:If you intend to use a subclass which modifies the apparent indices or values of this class with pytool.json.as_json(), remember to override for_json() to produce the data you desire.

Example:

from pytool.proxy import ListProxy

class SquaredList(ListProxy):
    def __setitem__(self, index, value):
        if isinstance(value, int):
            value *= value
        super(SquaredList, self).__setitem__(index, value)

my_list = range(5)
my_proxy = SquaredList(my_list)
my_proxy[3] = 5

my_proxy[3] # 25
my_list[3] # 25