Python
Notes, libraries, and links related to Python — tools I’ve used, want to try, or want to remember.
Python is a high-level, general-purpose programming language. Its design philosophy emphasizes code readability, most notably through the use of significant indentation.
Libraries
Core & Tooling
- Typer — Build CLI applications that are pleasant to use and maintain.
🐙 - py-spy — Sampling profiler for running Python programs.
- cleanco — Cleans company names by stripping legal suffixes.
🐍 - sections — Tree-like data structures for organizing strings and sections.
🐙 · 🐍
Data Access & Persistence
- asyncpg — High-performance Postgres client for asyncio.
- dataset — Databases for lazy people.
🐙 · 🐍 - Datafiles — File-based ORM for dataclasses.
🐙 · 🐍 - PyArrow — Python bindings for Apache Arrow.
🐍
Parsing, Extraction & Formats
GraphQL
- tartiflette — GraphQL engine built around the Schema Definition Language.
🐙 · 🐍
LLMs & ML Infrastructure
- LlamaIndex — Framework for ingesting and querying private or domain-specific data with LLMs.
- yaps — Surface language for programming Stan models using Python syntax.
🐙
Data Science
- datacompy — Pandas and Spark DataFrame comparison.
🐙 · 🐍 - intake — Plugin system for loading data and building catalogs.
🐙 · 🐍 - locopy — Load/unload data to Amazon Redshift.
🐙 · 🐍 - Luigi — Build complex pipelines of batch jobs.
🐙 · 🐍 - Vaex — Explore and visualize billion-row tabular datasets.
📑 · 🐙 · 🐍 · 📢 - NFStream — Data structures for online and offline network traffic analysis.
🐙 · 🐍
Snippets
*args and **kwargs?
How To Use args and kwargs in Python 3 by Lisa Tagliaferri, 20 November 2017.
>>> def hello(*args, **kwargs):
... gretting = kwargs.pop('gretting', 'Hello')
...
... print(f"""{gretting} {' '.join(args)}!""")
...
>>>
>>> hello("Laura", "Dang", gretting="Hi")
Hi Laura Dang!
Bare asterisk (*) in function argument
In Python 3 you can specify * in the argument list, from the Python 3.5 Documentation:
”Parameters after
*or*identifierare keyword-only parameters and may only be passed used keyword arguments.”
>>> def is_birthday(*, birthday):
... today = datetime.date.today()
... if birthday.month == today.month and birthday.day == today.day:
... print(f"Yes it's their birthday.")
... else:
... print(f"No it's not their birthday.")
...
>>>
>>> is_birthday(datetime.date(1986, 9, 19))
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: is_birthday() takes exactly 1 positional argument (1 given)
>>>
>>> is_birthday(birthday=datetime.date(1986, 9, 19))
No it's not their birthday.
>>>
Forced naming of parameters in Python by Eli Bendersky, 12 January 2013.
Coerce to NamedTuple
from typing import Any, Dict, NamedTuple, Type
class AlbumTuple(NamedTuple):
"""Album Tuple Class."""
name: str
artist: str
length: float
def coerce_to_namedtuple(d: Dict[str, Any], T: Type):
"""Create a NamedTuple from a dict, ignoring extra keys in the dict"""
return T(**{k: v for k, v in d.items() if k in T._fields})
album = dict(name="Liar", artist="Fake Shark - Real Zombie!", length=47.15)
print(coerce_to_namedtuple(d, AlbumTuple))
coerce_to_namedtuple.py by Andy Mitchhank
Links
- Python is two languages now, and that’s actually great — Tin, 27 Feb 2023
- Extracting Text Made Easy with Textract — Yancy Dennis, 19 Feb 2023
- Detecting SQL injections in Python code using AST — Artem Golubin, 29 Apr 2019
- Dive into Machine Learning
- Einstein Summation in NumPy — Olexa Bilaniuk, 4 Feb 2016
- My Python Development Environment — Jacob Kaplan-Moss, 21 Feb 2018
- Named Entity Recognition with NLTK and spaCy — Susan Li, 17 Aug 2018
- Python at Netflix — 29 Apr 2019
- Text Preprocessing in Python — Olga Davydova, 15 Oct 2018
- Working efficiently with JupyterLab Notebooks — Florian Wilhelm, 8 Nov 2018
- wtfpython — Surprising Python snippets and edge cases
- Understanding the asterisk (*) in Python — mingrammer, 20 Mar 2017
- An oral history of bank Python — Cal Paterson, Nov 2021
The strange world of Python inside big investment banks. - How we optimized a Python API server 100× — Vadim Markovtsev, 9 Feb 2022
- Refactoring a Python Codebase with LibCST — Adam Stepinski, 18 Feb 2022
- The Right Way to Compare Floats in Python — David Amos, 21 Mar 2022