Art from compromise: pyCustoms inspects Python packages

Consider technical computing.  Matlab is expensive but simple: One function per .m file – send a function inputs, get outputs.  Python’s adherents claim that it can supplant Matlab for most scientific purposes.  Reality, as usual, is more nuanced.  Since Python supports objects, classes, namespaces, and a lot of other funky features, Python tools are chock full of them.  Pick a package – numpy, scipy, matplotlib, or any of the ‘batteries included’ standard library.  It is difficult to figure out how to pass inputs to something and get outputs, assuming that thing is a function and not an object with methods, a class, a module,  or something else.  Documentation is often lacking so there will be multiple visits to StackOverflow, Usenet and Google Groups, and mailing lists.

I wrote some experimental Python spaghetti code to take a Python package, figure out which of its modules connect to which other modules, and then to recursively list each module’s builtins, classes, functions, submodules, and a bunch of stuff falling into ‘none of the above.’  I also sent the results into graphviz to visualize the results and perhaps gain some insight.  It was one compromise after another, figuring out ‘good enough’ when ‘ideal’ wasn’t convenient or  possible.  The firework-like graphviz output was fun to look at although not practically useful due to the large amount of zooming and panning needed to see details – what you see is all you’ve got.  I may use the plain text output from the pyCustoms algorithm in the future to figure out the lay of the land before studying a package in any detail.

The pyCustoms code is on Github in a Jupyter Notebook.  Here are the graphviz outputs for numpy and matplotlib.  Each image links to a PDF.  Zooming and panning works better in a standalone PDF reader than in a typical browser PDF plugin.  Right-clicking should permit downloading the files.  I normally use the Skim PDF reader for Macs but was surprised to find that Acrobat DC did a better job for these graphics intensive files.



Leave a Reply

Your email address will not be published. Required fields are marked *