Item — HN Reader

> But Python encourages the direct opposite - writing long files of code to avoid introducing too many "modules" and this is a debilitating headache.

How does Python encourage mega-files? AFAIK there's nothing that prevents nor discourages proper organization of code into files.

I frequently use multiple files to organize my library implementations.

1 comments

Just some clues: A class definition can not be split in many files (e.g. in C# you can do this easily with "partial class" declaration and that's amazing, needless to say you can and almost always do split one module (a namespace) into many flies there). Whatever you put in a separate file you will have to import manually everywhere you use it and you have to care about circular imports. You have to invent names all the time, a package, a module and a function having the same name smells a headache. The C# namespaces definition system could be made even better than it is already (i.e. by allowing class-free functions directly inside namespaces) but it already feels just so much better than Python "module = file" model.

>A class definition can not be split in many files (e.g. in C# you can do this easily with "partial class" declaration and that's amazing, needless to say you can and almost always do split one module (a namespace) into many flies there).

I think most would consider this a good thing. WYSIWYG. If a class is so gargantuan as to require multiple files, it should probably be multiple classes that communicate via a well defined, public API, not implicitly.

>You have to invent names all the time, a package, a module and a function having the same name smells a headache.

This is only an issue if you are importing *. If you import in a namespace (`from foo import module`, `from bar import other_module`), you can have `module.module`, `other_module.module` and anything else all living in harmony. If you work in python as python intends and not C/C++ (namespaced imports, not text-prepended ones), it's much, much cleaner.

It's also possible to, generally speaking, have private submodules within one larger module. You can do this with `__init__.py` by importing the various submodules, explicitly re-exporting the things you wish to be top-level public, and setting `__all__` to include them. This is why I can do something like `import numpy as np; np.ndarray` even though ndarray is defined in an extension module referenced by `numpy.core.multiarray`.

See multiarray (https://github.com/numpy/numpy/blob/master/numpy/core/multia...), which pulls things from `numpy.core._multiarray_umath`, then exported by core via `numeric` (https://github.com/numpy/numpy/blob/master/numpy/core/__init...), then again by the top level __init__ (https://github.com/numpy/numpy/blob/master/numpy/__init__.py...).

> If a class is so gargantuan as to require multiple files

IMHO anything that doesn't fit in one screen is "gargantuan"

> If you import in a namespace (`from foo import module`, `from bar import other_module`), you can have `module.module`, `other_module.module`

Which looks ugly and feels a nasty headache if you want to maintain intuitive vision of your code and dependencies structure.

> It's also possible to, generally speaking, have private submodules within one larger module. You can do this with `__init__.py` by importing the various submodules, explicitly re-exporting

I know but even reading this paragraph hurts. Too much mess to manage manually.

>IMHO anything that doesn't fit in one screen is "gargantuan"

This is a bit of an odd definition, but sure (I've seen C++ files whose imports alone were gargantuan under your definition). I just looked through a production python application, and it there were 2 non-test classes that were more than 100 lines long. Both were relatively verbose (well commented, use type hints so argument lists use significant vertical space, etc.)

In general, it seems like you're limiting yourself to, with the necessary boilerplate, one or two functions in any file that isn't "gargantuan". This makes it needlessly difficult to understand the structure of your applications since you are forced to split related logic up among multiple files. This, more than screen length or the import syntax, makes it much, much harder to "maintain intuitive vision of code and dependencies structure" as you say.

>Which looks ugly and feels a nasty headache if you want to maintain intuitive vision of your code and dependencies structure.

Not at all. It's more explicit about the dependency structure (`module.method(args)` at a call site gives you a much better idea of the structure than just `method(args)`), and so makes it significantly easier to maintain an intuitive vision of the code and dependencies.

It's incalculably easier to understand dependency structure when using the `import module` syntax than `from module import Class` or `from module import *`. So much so that the former allows reliable, large scale refactorings without any runtime information. The others, as you correctly recognize, do not.

>I know but even reading this paragraph hurts. Too much mess to manage manually.

Then don't! You don't need to. It's really only necessary for truly public/widely used APIs (like numpy) where understanding the internal structure of the module is not worth it for the average user.

> Then don't! You don't need to. It's really only necessary for truly public/widely used APIs (like numpy) where understanding the internal structure of the module is not worth it for the average user.

I mean for the library developer, not for the user.