pytool: Python helper package¶
This package contains a lot of helpful little methods and functions to make your life a little easier and your day a little better. Enjoy!
If you would like to see something added here, feel free to open up a GitHub issue.
Contents
- pytool: Python helper package
pytool.cmd
: Command helperspytool.json
: JSON helperspytool.lang
: Language helperspytool.text
: Text helperspytool.time
: Time and Date related helperspytool.proxy
: DictProxy and ListProxy classes
pytool.cmd
: Command helpers¶
This module contains helpers related to writing scripts and creating command line utilities.
Command helpers
pytool.cmd.Command
¶
-
class
pytool.cmd.
Command
¶ Base class for creating commands that can be run easily as scripts. This class is designed to be used with the
console_scripts
entry point to create Python-based commands for your packages.New in version 3.11.0: If the configargparse library is installed, Pytool will automatically use that as a drop-in replacement for the stdlib argparse module that is used by default.
You should use
parser_opts()
to give additional configuration arguments if you want to enable configargparse features like automatically using environment variables.Hello world example:
# hello.py from pytool.cmd import Command class HelloWorld(Command): def run(self): print "Hello World."
The only thing that must be defined in the subclass is the
run()
method, which should contain the code to launch your application, all other methods are optional.Example setup.py:
# setup.py from setuptools import setup setup( # ... entry_points={ 'console_scripts':[ 'helloworld = hello:HelloWorld.console_script', ], }, )
When using an entry point script, the
Command
has a specialconsole_script()
method for launching the application.Starting without an entry point script:
# hello.py [cont'd] if __name__ == '__main__': import sys HelloWorld().start(sys.argv[1:])
The
start()
method always requires an argument - even if it’s just an empty list.More complex example:
from pytool.cmd import Command class HelloAll(Command): def set_opts(self): self.opt('--world', default='World', help="use a different " "world") self.opt('--verbose', '-v', action='store_true', help="use " "more verbose output") def run(self): print "Hello", self.args.world if self.args.verbose: print "Hola", self.args.world
Whenever there are arguments for a command, they’re made available for your use as
self.args
. This object is created byargparse
so refer to that documentation for more information.-
classmethod
console_script
()¶ Method used to start the command when launched from a distutils console script.
-
describe
(description)¶ Describe the command in more detail. This will be displayed in addition to the argument help.
This automatically strips leading indentation but does not strip all formatting like the
ArgumentParser(description='')
keyword.Example:
class MyCommand(Command): def set_opts(self): self.describe(""" This is an example command. To use the example command, run it.""") def run(self): pass
-
opt
(*args, **kwargs)¶ Add an option to this command. This takes the same arguments as
ArgumentParser.add_argument()
.
-
parser_opts
()¶ Subclasses should override this method to return a dictionary of additional arguments to the parser instance.
Example:
class MyCommand(Command): def parser_opts(self): return dict( description="Manual description for cmd.", add_env_var_help=True, auto_env_var_prefix='mycmd_', )
-
reload
()¶ Reloads pyconfig if it is available.
Override this in your subclass if you wish to implement different reloading behavior.
-
run
()¶ Subclasses should override this method to start the command process. In other words, this is where the magic happens.
Changed in version 3.15.0: By default, this will just print help and exit.
-
set_opts
()¶ Subclasses should override this method to configure the command line arguments and options.
Example:
class MyCommand(Command): def set_opts(self): self.opt('--verbose', '-v', action='store_true', help="be more verbose") def run(self): if self.args.verbose: print "I'm verbose."
-
start
(args)¶ Starts a command and registers single handlers.
-
stop
(*args, **kwargs)¶ Exits the currently running process with status 0.
Override this in your subclass if you wish to implement different SIGINT or SIGTERM handling for your process.
-
subcommand
(name, opt_func=None, run_func=None, *args, **kwargs)¶ Add a subcommand name with setup opt_func and main run_func to the argument parser.
Any additional positional or keyword arguments will be passed to the
ArgumentParser
instance created.Changed in version 3.15.0: Either opt_func or run_func may be omitted, in which case a method with a name matching the subcommand name plus
_opts
will be bound to opt_func and a method matching the subcommand name will be bound to run_func.For example, a subcommand
'write'
will bind the methodswrite_opts()
andwrite()
.New in version 3.12.0.
Example:
class MyCommand(Command): def set_opts(self): self.subcommand('thing', self.thing, self.run_thing) def thing(self): # Set thing specific options self.opt('--thing', 't', help="Thing to do") def run_thing(self): # This runs if the thing subcommand is invoked pass def run(self): # This runs if there is no subcommand
Parameters: - name (str) – Subcommand name
- opt_func (function) – Options function to add
- run_func (function) – Run function to add
- args – Arguments to pass to the subparser constructor
- kwargs – Keyword arguments to pass to the subparser constructor
-
classmethod
pytool.json
: JSON helpers¶
This module contains helpers for working with JSON data.
Tries to use the simplejson module if it exists, otherwise falls back to the json module.
If the bson module exists, it allows bson.ObjectId objects to be decoded into JSON automatically.
JSON helpers
as_json()
¶
-
pytool.json.
as_json
(obj, **kwargs)¶ Returns an object JSON encoded properly.
This method allows you to implement a hook method
for_json()
on your objects if you want to allow arbitrary objects to be encoded to JSON. Afor_json()
hook must return a basic JSON type (dict, list, int, float, string, unicode, float or None), or a basic JSON type which contains other objects which implement thefor_json()
hook.If an object implements both
_asdict()
andfor_json()
the latter is given preference.Also adds additional encoders for
datetime
andbson.ObjectId
.Parameters: - obj (object) – An object to encode.
- **kwargs –
Any optional keyword arguments to pass to the JSONEncoder
Returns: JSON encoded version of obj.
New in version 2.4: Objects which have an
_asdict()
method will have that method called as part of encoding to JSON, even when not using simplejson.New in version 2.4: Objects which have a
for_json()
method will have that method called and the return value used for encoding instead.Changed in version 3.0: simplejson (
>= 3.2.0
) is now required, and relied upon for the_asdict()
andfor_json()
hooks. This change may break backwards compatibility in any code that uses these hooks.
from_json()
¶
-
pytool.json.
from_json
(value)¶ Decodes a JSON string into an object.
Parameters: value (str) – String to decode Returns: Decoded JSON object
pytool.lang
: Language helpers¶
This module contains items that are “missing” from the Python standard library, that do miscelleneous things.
Language helpers
Namespace
¶
-
class
pytool.lang.
Namespace
(obj=None)¶ Namespace object used for creating arbitrary data spaces. This can be used to create nested namespace objects. It can represent itself as a dictionary of dot notation keys.
Basic usage:
>>> from pytool.lang import Namespace >>> # Namespaces automatically nest >>> myns = Namespace() >>> myns.hello = 'world' >>> myns.example.value = True >>> # Namespaces can be converted to dictionaries >>> myns.as_dict() {'hello': 'world', 'example.value': True} >>> # Namespaces have container syntax >>> 'hello' in myns True >>> 'example.value' in myns True >>> 'example.banana' in myns False >>> 'example' in myns True >>> # Namespaces are iterable >>> for name, value in myns: ... print name, value ... hello world example.value True >>> # Namespaces that are empty evaluate as False >>> bool(Namespace()) False >>> bool(myns.notset) False >>> bool(myns) True >>> # Namespaces allow the __get__ portion of the descriptor protocol >>> # to work on instances (normally they would not) >>> class MyDescriptor(object): ... def __get__(self, instance, owner): ... return 'Hello World' ... >>> myns.descriptor = MyDescriptor() >>> myns.descriptor 'Hello World' >>> # Namespaces can be created from dictionaries >>> newns = Namespace({'foo': {'bar': 1}}) >>> newns.foo.bar 1 >>> # Namespaces will expand dot-notation dictionaries >>> dotns = Namespace({'foo.bar': 2}) >>> dotns.foo.bar 2 >>> # Namespaces will coerce list-like dictionaries into lists >>> listns = Namespace({'listish': {'0': 'zero', '1': 'one'}}) >>> listns.listish ['zero', 'one'] >>> # Namespaces can be deepcopied >>> a = Namespace({'foo': [[1, 2], 3]} >>> b = a.copy() >>> b.foo[0][0] = 9 >>> a.foo [[1, 2], 3] >>> b.foo [[9, 2], 3] >>> # You can access keys using dict-like syntax, which is useful >>> myns.foo.bar = True >>> myns['foo'].bar True >>> # Dict-like access lets you traverse namespaces >>> myns['foo.bar'] True >>> # Dict-like access lets you traverse lists as well >>> listns['listish.0'] 'zero' >>> listns['listish.1'] 'one' >>> # Dict-like access lets you traverse nested lists and namespaces >>> nested = Namespace() >>> nested.values = [] >>> nested.values.append(Namespace({'foo': 'bar'})) >>> nested['values.0.foo'] 'bar' >>> # You can also call the traversal method if you need >>> nested.traversal(['values', 0, 'foo']) 'bar'
Namespaces are useful!
New in version 3.5.0: Added the ability to create Namespace instances from dictionaries.
New in version 3.6.0: Added the ability to handle dot-notation keys and list-like dicts.
New in version 3.7.0: Added deepcopy capability to Namespaces.
New in version 3.8.0: Added dict-like access capability to Namespaces.
New in version 3.9.0: Added traversal by key/index arrays for nested Namespaces and lists
-
as_dict
(base_name=None)¶ Return the current namespace as a dictionary.
Parameters: base_name (str) – Base namespace (optional)
-
copy
(*args, **kwargs)¶ Return a copy of a Namespace by writing it to a dict and then writing back to a Namespace.
Arguments to this method are ignored.
-
from_dict
(obj)¶ Populate this Namespace from the given obj dictionary.
Parameters: obj (dict) – Dictionary object to merge into this Namespace New in version 3.5.0.
-
items
(base_name=None)¶ Return generator which returns
(key, value)
tuples.Analagous to dict.items() behavior in Python3
Parameters: base_name (str) – Base namespace (optional)
-
iteritems
(base_name=None)¶ Return generator which returns
(key, value)
tuples.Parameters: base_name (str) – Base namespace (optional)
-
traverse
(path)¶ - Traverse the Namespace and any nested elements by following the
elements in an iterable path and return the item found at the end of path.
Traversal is achieved using the __getitem__ method, allowing for traversal of nested structures such as arrays and dictionaries.
AttributeError is raised if one of the attributes in path does not exist at the expected depth.
param iterable path: An iterable whose elements specify the keys to path over.
Example:
- ns = Namespace({“foo”:
- [Namespace({“name”: “john”}),
- Namespace({“name”: “jane”})]})
ns.traverse([“foo”, 1, “name”])
… returns …
“jane”
-
UNSET
¶
-
class
pytool.lang.
UNSET
¶ Special class that evaluates to
bool(False)
, but can be distinctly identified as seperate fromNone
orFalse
. This class can and should be used without instantiation.>>> from pytool.lang import UNSET >>> # Evaluates to False >>> bool(UNSET) False >>> # Is a class-singleton (cannot become an instance) >>> UNSET() is UNSET True >>> # Is good for checking default values >>> if {}.get('example', UNSET) is UNSET: ... print "Key is missing." ... Key is missing. >>> # Has no length >>> len(UNSET) 0 >>> # Is iterable, but has no iterations >>> list(UNSET) [] >>> # It has a repr() equal to itself >>> UNSET UNSET
classproperty()
¶
-
pytool.lang.
classproperty
(func)¶ Makes a
@classmethod
style property (since@property
only works on instances).from pytool.lang import classproperty class MyClass(object): _attr = 'Hello World' @classproperty def attr(cls): return cls._attr MyClass.attr # 'Hello World' MyClass().attr # Still 'Hello World'
get_name()
¶
-
pytool.lang.
get_name
(frame)¶ Gets the name of the passed frame.
Warning: It’s very important to delete a stack frame after you’re done using it, as it can cause circular references that prevents garbage collection. Parameters: frame – Stack frame to inspect. Returns: Name of the frame in the form module.class.method.
hashed_singleton()
¶
-
pytool.lang.
hashed_singleton
(klass)¶ Wraps a class to create a hashed singleton version of it. A hashed singleton is like a singleton in that there will be only a single instance of the class for each call signature.
The singleton is kept as a weak reference, so if your program ceases to reference the hashed singleton, you may get a new instance if the Python interpreter has garbage collected your original instance.
This will not work for classes that take arguments that are unhashable (e.g. dicts, sets).
Parameters: klass – Class to decorate New in version 2.1.
Changed in version 3.4.2: @hashed_singleton wrapped classes now preserve their @staticmethod functions on the class type as well as the instance.
Example usage:
# Make a class directly behave as a hashed singleton @hashed_singleton class Test(object): def __init__(self, *args, **kwargs): pass # Make an imported class behave as a hashed singleton Test = hashed_singleton(Test) # The same arguments give you the same class instance back test = Test('a', k='k') test is Test('a', k='k') # True # A different argument signature will give you a new instance test is Test('b', k='k') # False test is Test('a', k='j') # False # Removing all references to a hashed singleton instance will allow # it to be garbage collected like normal, because it's only kept # as a weak reference del test test = Test('a', k='k') # If the Python interpreter has garbage # collected, you will get a new instance
singleton()
¶
-
pytool.lang.
singleton
(klass)¶ Wraps a class to create a singleton version of it.
Parameters: klass – Class to decorate Changed in version 3.4.2: @singleton wrapped classes now preserve their @staticmethod functions on the class type as well as the instance.
Example usage:
# Make a class directly behave as a singleton @singleton class Test(object): pass # Make an imported class behave as a singleton Test = singleton(Test)
unflatten()
¶
-
pytool.lang.
unflatten
(obj)¶ Return obj with dot-notation keys unflattened into nested dictionaries, as well as list-like dictionaries converted into list instances.
Parameters: obj – An arbitrary object, preferably a dict
pytool.text
: Text helpers¶
This module contains text related things that make life easier.
wrap()
¶
-
pytool.text.
wrap
(text, width=None, indent='')¶ Return text wrapped to width while trimming leading indentation and preserving paragraphs.
This function is handy for turning indented inline strings into unindented strings that preserve paragraphs, whitespace, and any indentation beyond the baseline.
The default wrap width is the number of columns available in the current console or TTY. If the columns can’t be found, then the default wrap width is
79
.Parameters: - text (str) – Text to wrap
- width (int) – Width to wrap text at (default: text column width)
- indent (str) – String to indent text with (default: ‘’)
>>> import pytool >>> text = ''' All this is indented by 8, but will be 0. This is indented by 16, and a really long long long line which is hard wrapped at a random character width, but will be wrapped appropriately at 70 chars afterwards. This is indented by 8 again. ''' >>> print pytool.text.wrap(text) All this is indented by 8, but will be 0. This is indented by 16, and a really long long long line which is hard wrapped at a random character width, but will be wrapped appropriately at 70 chars afterwards. This is indented by 8 again. >>>
pytool.time
: Time and Date related helpers¶
This module contains time related things that make life easier.
Timer
¶
-
class
pytool.time.
Timer
¶ This is a simple timer class.
timer = pytool.time.Timer() for i in (1, 2, 3): sleep(i) print timer.mark(), "elapsed since last mark or start" print timer.elapsed, "total elapsed"
-
elapsed
¶ Return a
timedelta
of the time elapsed since the start.
-
mark
()¶ Return a
timedelta
of the time elapsed since the last mark or start.
-
UTC
¶
-
class
pytool.time.
UTC
¶ UTC timezone. This is necessary since Python doesn’t include any explicit timezone objects in the standard library. This can be used to create timezone-aware datetime objects, which are a pain to work with, but a necessary evil sometimes.
from datetime import datetime from pytool.time import UTC utc_now = datetime.now(UTC())
as_utc()
¶
-
pytool.time.
as_utc
(stamp)¶ Converts any datetime (naive or aware) to UTC time.
Parameters: stamp (datetime) – Datetime to convert Returns: stamp as UTC time from datetime import datetime from pytool.time import as_utc utc_datetime = as_utc(datetime.now())
utcnow()
¶
-
pytool.time.
utcnow
()¶ Return the current UTC time as a timezone-aware datetime.
Returns: The current UTC time
fromutctimestamp()
¶
-
pytool.time.
fromutctimestamp
(stamp)¶ Return a timezone-aware datetime object from a UTC unix timestamp.
Parameters: stamp (float) – Unix timestamp in UTC Returns: UTC datetime object import time from pytool.time import fromutctimestamp utc_datetime = fromutctimestamp(time.time())
toutctimestamp()
¶
-
pytool.time.
toutctimestamp
(stamp)¶ Converts a naive datetime object to a UTC unix timestamp. This has an advantage over time.mktime in that it preserves the decimal portion of the timestamp when converting.
Parameters: stamp (datetime) – Datetime to convert Returns: Unix timestamp as a float
from datetime import datetime from pytool.time import toutctimestamp utc_stamp = toutctimestamp(datetime.now())
is_dst()
¶
-
pytool.time.
is_dst
(stamp)¶ Return
True
if stamp is daylight savings.Parameters: stamp (datetime) – Datetime Returns: True
if stamp is daylight savings, otherwiseFalse
.
trim_time()
¶
-
pytool.time.
trim_time
(stamp)¶ Trims the time portion off of stamp, leaving the date intact. Returns a datetime of the same date, set to 00:00:00 hours. Preserves timezone information.
Parameters: stamp (datetime) – Timestamp to trim Returns: Trimmed timestamp
floor_day()
¶
-
pytool.time.
floor_day
(stamp=None)¶ Return stamp floored to the current day. If no stamp is specified, the current time is used. This is similar to the
date()
method, but returns adatetime
object, instead of adate
object.This is the same as
trim_time()
.Changed in version 2.0: Preserves timezone information if it exists, and uses
pytool.time.utcnow()
instead ofdatetime.datetime.now()
if stamp is not given.Parameters: stamp (datetime) – datetime object to floor (default: now) Returns: Datetime floored to the day
floor_minute()
¶
-
pytool.time.
floor_minute
(stamp=None)¶ Return stamp floored to the current minute. If no stamp is specified, the current time is used. Preserves timezone information.
New in version 2.0.
Parameters: stamp (datetime) – datetime object to floor (default: now) Returns: Datetime floored to the minute
floor_month()
¶
-
pytool.time.
floor_month
(stamp=None)¶ Return stamp floored to the current month. If no stamp is specified, the current time is used.
Changed in version 2.0: Preserves timezone information if it exists, and uses
pytool.time.utcnow()
instead ofdatetime.datetime.now()
if stamp is not given.Parameters: stamp (datetime) – datetime object to floor (default: now) Returns: Datetime floored to the month
floor_week()
¶
-
pytool.time.
floor_week
(stamp=None)¶ Return stamp floored to the current week, at 00:00 Monday. If no stamp is specified, the current time is used. Preserves timezone information.
This is the same as
week_start()
New in version 2.0.
Parameters: stamp (datetime) – datetime object to floor (default now:) Returns: Datetime floored to the week
make_week_seconds()
¶
-
pytool.time.
make_week_seconds
(day, hour, minute=0, seconds=0)¶ Return
week_seconds()
for the given day of the week, hour and minute.Parameters: - day (int) – Zero-indexed day of the week
- hour (int) – Zero-indexed 24-hour
- minute (int) – Minute (default:
0
) - seconds (int) – Seconds (default:
0
)
Returns: Seconds since 00:00 Monday
week_seconds()
¶
-
pytool.time.
week_seconds
(stamp)¶ Return stamp converted to seconds since 00:00 Monday.
Parameters: stamp (datetime) – Timestamp to convert Returns: Seconds since 00:00 monday
week_seconds_to_datetime()
¶
-
pytool.time.
week_seconds_to_datetime
(seconds)¶ Return the datetime that is seconds from the start of this week.
Parameters: seconds (int) – Seconds Returns: Datetime for 00:00 Monday plus seconds
week_start()
¶
-
pytool.time.
week_start
(stamp)¶ Return the start of the week containing stamp.
Changed in version 2.0: Preserves timezone information.
Parameters: stamp (datetime) – Timestamp Returns: A datetime for 00:00 Monday of the given week
ago()
¶
-
pytool.time.
ago
(stamp=None, **kwargs)¶ Return the current time as UTC minutes the specified timeframe.
This is a helper for simplifying the common pattern of pytool.time.utcnow() - datetime.timedelta(mintues=15).
Parameters: - stamp – An optional timestamp instead of utcnow()
- days – Days previous
- hours – Hours previous
- minutes – Minutes previous
- seconds – Days previous
Returns: UTC timestamp
If you like brevity in your arguments, you can use the shorter versions, hrs=, mins= and secs=.
Or if you really want a short signature, you can use d=, h=, m=, s=.
import pytool yesterday = pytool.time.ago(days=1) a_little_while_ago = pytool.time.ago(minutes=1) a_little_before = pytool.time.ago(my_date, hours=1) # Shorter arguments shorter = pytool.time.ago(hrs=1, mins=1, secs=1) # Shorthand argument names short = pytool.time.ago(d=1, h=1, m=1, s=1)
pytool.proxy
: DictProxy and ListProxy classes¶
This module contains implmentations of proxy-list and proxy-dictionary objects.
DictProxy
¶
-
class
pytool.proxy.
DictProxy
(data)¶ Proxies all methods for a dict instance.
This is useful when you want to modify a dictionary’s behavior through subclassing without copying the dictionary or if you want to be able to modify the original dictionary.
Parameters: data – A dict or dict-like object (implements all the collections.MutableMapping
methods)New in version 2.2.
Note: If you intend to use a subclass which modifies the apparent keys or values of this class with pytool.json.as_json()
, remember to overridefor_json()
to produce the data you desire.Example:
from pytool.proxy import DictProxy class SquaredDict(DictProxy): def __getitem__(self, key): value = super(SquaredDict, self).__getitem__(key) if isinstance(value, int): value *= value return value my_dict = {} my_proxy = SquaredDict(my_dict) my_proxy['val'] = 5 my_proxy['val'] # 25 my_dict['val'] # 5
-
clear
() → None. Remove all items from D.¶
-
get
(k[, d]) → D[k] if k in D, else d. d defaults to None.¶
-
items
() → list of D's (key, value) pairs, as 2-tuples¶
-
iteritems
() → an iterator over the (key, value) items of D¶
-
iterkeys
() → an iterator over the keys of D¶
-
itervalues
() → an iterator over the values of D¶
-
keys
() → list of D's keys¶
-
pop
(k[, d]) → v, remove specified key and return the corresponding value.¶ If key is not found, d is returned if given, otherwise KeyError is raised.
-
popitem
() → (k, v), remove and return some (key, value) pair¶ as a 2-tuple; but raise KeyError if D is empty.
-
setdefault
(k[, d]) → D.get(k,d), also set D[k]=d if k not in D¶
-
update
([E, ]**F) → None. Update D from mapping/iterable E and F.¶ If E present and has a .keys() method, does: for k in E: D[k] = E[k] If E present and lacks .keys() method, does: for (k, v) in E: D[k] = v In either case, this is followed by: for k, v in F.items(): D[k] = v
-
values
() → list of D's values¶
-
ListProxy
¶
-
class
pytool.proxy.
ListProxy
(data)¶ Proxies all methods for a list instance. This is useful when you want to modify a list’s behavior without copying the list.
Methods which do not mutate a list, and instead return a new list will return a list instance rather than a ListProxy instance.
Parameters: data – A list or list-like object (implements all the collections.MutableSequence
methods)New in version 2.2.
Note: If you intend to use a subclass which modifies the apparent indices or values of this class with pytool.json.as_json()
, remember to overridefor_json()
to produce the data you desire.Example:
from pytool.proxy import ListProxy class SquaredList(ListProxy): def __setitem__(self, index, value): if isinstance(value, int): value *= value super(SquaredList, self).__setitem__(index, value) my_list = range(5) my_proxy = SquaredList(my_list) my_proxy[3] = 5 my_proxy[3] # 25 my_list[3] # 25
-
append
(item)¶ S.append(object) – append object to the end of the sequence
-
count
(value) → integer -- return number of occurrences of value¶
-
extend
(other)¶ S.extend(iterable) – extend sequence by appending elements from the iterable
-
index
(value) → integer -- return first index of value.¶ Raises ValueError if the value is not present.
-
insert
(i, item)¶ S.insert(index, object) – insert object before index
-
pop
([index]) → item -- remove and return item at index (default last).¶ Raise IndexError if list is empty or index is out of range.
-
remove
(item)¶ S.remove(value) – remove first occurrence of value. Raise ValueError if the value is not present.
-
reverse
()¶ S.reverse() – reverse IN PLACE
-