-
-
Notifications
You must be signed in to change notification settings - Fork 784
Initial working draft of File Management guide. #3552
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
@freakboy3742 This is definitely still a draft, but I've reached a point where I believe more direct feedback is needed. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Totally understandable things for a first draft. A very useful addition for Toga! It might be worth linking this into the BeeWare tutorial list of topics. Can't wait for a final version!
| Python defaults to looking in the current working directory (the directory from which the script is being run), and all path access from this is implicit. You are able to use relative paths to point to a location. This might even work with simple Toga scripts. However, when you get into packaging applications, things change. | ||
|
|
||
| A packaged app can be run from anywhere on a computer, and there's no guarantee you can write a file to the location from which the app is being run. You can package file content into an app, but there's still no guarantee of the location it will be installed or run. Further, you may run into permissions issues attempting to write to the app location. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure, but might be worth mentioning that on macOS, writing a file breaks the signature and on iOS it'll flat-out crash if you're writing in the app bundle. In other words, to further stress the point, maybe use macOS and iOS as an example that you might not be able to write things using conventional methods, at all.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is mentioned later, but might be worth moving to the top a bit.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's true that iOS will crash; however, there's two issue here - one is construction of absolute paths; the second is the viability of the write location. The viability of the write location is a secondary concern (and one that only matters if you are actually writing a file - it doesn't affect reads.
This discussion of writability is covered much better in the section on writing files; I think mentioning writing here is a distraction. At this point we're purely worried about finding the file in the first place.
|
|
||
| Unlike relative paths, absolute paths do not rely on the location of the Python program or application. They provide the full path of the file. There are multiple ways to construct absolute paths in Python. | ||
|
|
||
| The most basic method is to spell out the entire path. For example, on macOS or Unix, to access ``file.txt`` in a directory called ``file_directory`` in your home directory, you could use ``"~/file_directory/file.txt"``. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| The most basic method is to spell out the entire path. For example, on macOS or Unix, to access ``file.txt`` in a directory called ``file_directory`` in your home directory, you could use ``"~/file_directory/file.txt"``. | |
| The most basic method is to spell out the entire path. For example, on macOS or Unix, to access ``file.txt`` in a directory called ``file_directory`` in your home directory, you could use ``~/file_directory/file.txt``. |
|
|
||
| The most basic method is to spell out the entire path. For example, on macOS or Unix, to access ``file.txt`` in a directory called ``file_directory`` in your home directory, you could use ``"~/file_directory/file.txt"``. | ||
|
|
||
| In any Python file, ``__file__`` is the full path to current working directory, i.e. the location of file being executed, provided as a string. We can use this to construct paths; for example, ``Path(__file__).parent`` is the parent directory of the currently running Python program. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Keep in mind that the location of the Python script, which is what __file__ provides, might not be the working directory. Try python random_dir/test.py and print __file__ inside.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the issue here is that "current working directory" is misleading. Firstly - __file__ won't ever be a directory - it will always be a file; but cwd means something in an execution context, and __file__ is a reference to the file being executed. The current wording is (mostly) true if you're running a single file through an interpreter, but won't be in the general case.
| One possible option is in the top level ``helloworld/`` directory, as that's the location from which we actually run the app. While we could point our code to this location as an absolute path, we will still run into the problem when running the app from anywhere else but our own computer. | ||
|
|
||
| A second possible option might be to put the file in ``helloworld/src/helloworld`` because that's where the ``app.py`` file is. After all, Python bases file access on the current working directory. This second option does ensure Briefcase packages the file with the app. However, apps can be run from anywhere on a computer, so it still doesn't guarantee a consistent path. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm sort of confused by the last sentence; that could use some clarifying.
|
|
||
| Toga includes an :doc:`app paths <../../reference/api/resources/app_paths>` feature that provides a selection of known locations on the user's computer. Provided as a ``pathlib.Path`` object, they are known-safe locations for reading and writing files, that are specific to each operating system. They are unique to each application, and guaranteed to be isolated to the specific app. There are four writable paths, and one read-only path. | ||
|
|
||
| The read-only path location, ``app``, provides an anchor from the location of the app file. It is the path of the directory that contains the Python file that defines the class that is being executed as the app, specifically the Python file that includes ``class MyApp(toga.app):``. In an application containing only a single file, is essentially returning ``Path(__file__).parent``. However, in an application with multiple levels of modules, or when calling a library that is independent of the app, calling ``Path(__file__)`` may not return the expected result, whereas ``app`` will return the same location no matter where it is. It can therefore be used to construct absolute paths based on the app file location within the package. For this to work, we need to package the file with our app. Briefcase guarantees that any file in the project directory (``helloworld/src/helloworld`` in the example project structure shown above), will be included with the packaged app, including the contents of any subdirectories. There are other ways to ensure a file is included - see the :doc:`Source <../../reference/api/resources/sources/source>` documentation for details. |
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No - the description here is correct. It literally resolves to the location of the file that contains the app class declaration.
However, the full depth of the description is something that might be better suited to a footnote; the core of the idea here is that paths.app provides a location that "describes the location of app code".
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This suggestion is incorrect
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(FYI referring to my suggestion)
|
|
||
| The read-only path location, ``app``, provides an anchor from the location of the app file. It is the path of the directory that contains the Python file that defines the class that is being executed as the app, specifically the Python file that includes ``class MyApp(toga.app):``. In an application containing only a single file, is essentially returning ``Path(__file__).parent``. However, in an application with multiple levels of modules, or when calling a library that is independent of the app, calling ``Path(__file__)`` may not return the expected result, whereas ``app`` will return the same location no matter where it is. It can therefore be used to construct absolute paths based on the app file location within the package. For this to work, we need to package the file with our app. Briefcase guarantees that any file in the project directory (``helloworld/src/helloworld`` in the example project structure shown above), will be included with the packaged app, including the contents of any subdirectories. There are other ways to ensure a file is included - see the :doc:`Source <../../reference/api/resources/sources/source>` documentation for details. | ||
|
|
||
| Let's build on the previous example to use the ``app`` to locate the file. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| Let's build on the previous example to use the ``app`` to locate the file. | |
| Let's build on the previous example to use ``App.paths.app`` to locate the file. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agreed this is a little ambiguous, but we need to find a way to refer to this concept consistently. I'm not sure App.paths.app is it, because thats not how most users will be using it.
| - ``cache``: The location for storing cache files. This should be used only for easily regenerated files as the operating system may purge the contents of this directory without warning. | ||
| - ``logs``: The location for storing log files. | ||
|
|
||
| These paths are different on every operating system, and Toga guarantees the correct paths will be provided. The paths will be subdirectories found in ``~/Library`` on macOS, XDG-compliant dotfiles in ``~`` on Linux, and ``AppData/`` on Windows. |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's not necessarily C:\users\.... I'd lean to "the user's AppData folder".
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The overall structure of this is great - it covers all the topics it needs to, and the example (with a couple of minor tweaks suggested inline) is a very clear exploration of the topic.
A couple of specific suggestions/corrections inline. I think the biggest area needing attention is the initial section, clarifying the specific sub-problems/misunderstandings that may exist:
- What is the difference between an absolute and relative path?
- How is a relative path resolved at runtime? (i.e., relative to CWD)
- What is the CWD when a bundled app runs? (i.e., who knows? Does it even make sense as a question?)
- If we can't rely on the CWD, what can we use? (
__file__is one option; but Toga has a better answer inpaths.app).
It might even be worth having a version that is based on Path(__file__).parent / "resources" / "initial_config.toml" - __file__ works, but in a more complex app, you would need to know where the file that is reading the code is in the code checkout relative to the resources folder.
| Python defaults to looking in the current working directory (the directory from which the script is being run), and all path access from this is implicit. You are able to use relative paths to point to a location. This might even work with simple Toga scripts. However, when you get into packaging applications, things change. | ||
|
|
||
| A packaged app can be run from anywhere on a computer, and there's no guarantee you can write a file to the location from which the app is being run. You can package file content into an app, but there's still no guarantee of the location it will be installed or run. Further, you may run into permissions issues attempting to write to the app location. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's true that iOS will crash; however, there's two issue here - one is construction of absolute paths; the second is the viability of the write location. The viability of the write location is a secondary concern (and one that only matters if you are actually writing a file - it doesn't affect reads.
This discussion of writability is covered much better in the section on writing files; I think mentioning writing here is a distraction. At this point we're purely worried about finding the file in the first place.
|
|
||
| The most basic method is to spell out the entire path. For example, on macOS or Unix, to access ``file.txt`` in a directory called ``file_directory`` in your home directory, you could use ``"~/file_directory/file.txt"``. | ||
|
|
||
| In any Python file, ``__file__`` is the full path to current working directory, i.e. the location of file being executed, provided as a string. We can use this to construct paths; for example, ``Path(__file__).parent`` is the parent directory of the currently running Python program. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the issue here is that "current working directory" is misleading. Firstly - __file__ won't ever be a directory - it will always be a file; but cwd means something in an execution context, and __file__ is a reference to the file being executed. The current wording is (mostly) true if you're running a single file through an interpreter, but won't be in the general case.
| - ``cache``: The location for storing cache files. This should be used only for easily regenerated files as the operating system may purge the contents of this directory without warning. | ||
| - ``logs``: The location for storing log files. | ||
|
|
||
| These paths are different on every operating system, and Toga guarantees the correct paths will be provided. The paths will be subdirectories found in ``~/Library`` on macOS, XDG-compliant dotfiles in ``~`` on Linux, and ``AppData/`` on Windows. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's not necessarily C:\users\.... I'd lean to "the user's AppData folder".
| This change adds a save button, that when pressed, saves the contents of the text input to a ``config.toml`` file in an app-specific subdirectory of the operating-system appropriate config directory, and displays the path to the file below the input. | ||
|
|
||
| Run the app and click the "load initial config" button to load the file contents into the text input. Update the variables to whatever you like. Click the save button to generate the file. Use the path displayed below the input to find and view your new config file. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is good advice, but might need a little more explicit instruction - they're going to need to do this with their file manager or terminal, or some other tool.
docs/spelling_wordlist
Outdated
| clickable | ||
| closable | ||
| codepoint | ||
| config |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we're referring to paths.config, then it should be quoted; if it's "the config file" then the English text should be expanded to "configuration" as appropriate.
docs/spelling_wordlist
Outdated
| Manjaro | ||
| misconfigured | ||
| Monetization | ||
| multiline |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should be either multi-line, or MultilineTextInput depending on context
docs/spelling_wordlist
Outdated
| prepending | ||
| programmatically | ||
| px | ||
| py |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This one seems odd; if it's needed, it suggests a deeper problem that we should investigate
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It fails with the following:
docs/how-to/topics/file-management.rst:7: : Spell check: py: For this guide, we're going to use Briefcase to package our application. However, the same path-related issues outlined here exist regardless of what tool you use to deploy your code, including Setuptools, py2app, PyInstaller, and so on..
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We can make a once-off exception by wrapping py2app with
:spelling:ignore:`py2app`
That will also require installing the sphinx spelling plugin (in conf.py) always - I'm not sure why we made it optional, but I can't see the harm.
Co-authored-by: Russell Keith-Magee <[email protected]>
Co-authored-by: Russell Keith-Magee <[email protected]>
Co-authored-by: Russell Keith-Magee <[email protected]>
Co-authored-by: Russell Keith-Magee <[email protected]>
Co-authored-by: Russell Keith-Magee <[email protected]>
@freakboy3742 I'm not particularly happy with the rework of the initial section, but I think I've done what I can. This may be to the point where you need to take over, but I'm here to complete anything else you think I should do. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A couple of ordering things and other minor cleanups flagged inline, but otherwise this is :chefs kiss:
|
|
||
| Relative paths might work with simple Toga scripts. However, when you get into packaging applications, things change. When you double-click on an icon to run an app, what directory is considered the current working directory? Is it the directory containing the app bundle? Is it the directory in the app bundle that holds the executable? There isn't a solid answer. The question we really need to answer is, how can we reliably tell an app where to look for file content? | ||
|
|
||
| If we can't rely on the current working directory to construct relative paths, what *can* we use? We need to use absolute paths. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This lead in is awesome - but it steals its own thunder. It introduces Absolute paths... and then talks about how absolute paths are the solution. The Absolute paths paragraph can probably be moved wholesale to be the introduction paragraph of the next section.
|
|
||
| The most basic method is to spell out the entire path. For example, on macOS or Unix, to access ``file.txt`` in a directory called ``file_directory`` in your home directory, you could use ``~/file_directory/file.txt``. | ||
|
|
||
| In any Python file, ``__file__`` resolves to the absolute path to the location of file being executed, provided as a string. We can use this to construct paths; for example, ``Path(__file__).parent`` is the parent directory of the currently running Python program. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Might want a lead in here like "Python also provides some tools to build absolute paths based on properties of the running python code."
|
|
||
| In any Python file, ``__file__`` resolves to the absolute path to the location of file being executed, provided as a string. We can use this to construct paths; for example, ``Path(__file__).parent`` is the parent directory of the currently running Python program. | ||
|
|
||
| The ``pathlib`` module, from the Python standard library, provides several absolute path options including ``Path.cwd()``, which is the path to the current working directory, and ``Path.home()``, which is the path to your home directory. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Part of this content needs to be tied into the previous paragraph, as it's not necessarily clear where Path() came from.
|
|
||
| Let's take a look at an example of reading file contents from a file using the basic method to construct an absolute path. | ||
|
|
||
| Create a new Briefcase project (for a refresher on how to do this, see `the BeeWare tutorial <https://docs.beeware.org/en/latest/tutorial/tutorial-1.html>`__). Once you've created that project, update ``app.py`` in the project to contain the following: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"... in this example, we've used a formal name of Config File Creator" (or similar), to give the lead in to how to drive Briefcase to give the following stub.
| class ConfigFileCreator(toga.App): | ||
| def startup(self): | ||
| self.text_input = toga.MultilineTextInput() | ||
| self.config_directory = toga.TextInput(readonly=True) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Minor nit - might want to name this config_directory_input, so it doesn't read as if it is the config directory.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(or a similar, shorter name...)
|
|
||
| Toga includes an :doc:`app paths <../../reference/api/resources/app_paths>` feature that provides a selection of known locations on the user's computer. Provided as ``pathlib.Path`` objects, they are known-safe locations for reading and writing files, that are specific to each operating system. Each user running an application will have their own unique app paths. | ||
|
|
||
| The read-only path location, ``paths.app``, provides an anchor from the location of the app file. [#f1]_ It can therefore be used to construct absolute paths based on the app file location within the package. For this to work, we need to package the file with our app. Briefcase guarantees that any file in the project directory (``configfilecreator/src/configfilecreator`` in the example project structure shown above), will be included with the packaged app, including the contents of any subdirectories. There are other ways to ensure a file is included - see the :doc:`Source <../../reference/api/resources/sources/source>` documentation for details. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The reference to Briefcase's packaging behavior has been referred to indirectly above; might want to restructure that section further up the page to the point where we're creating the initial_config.toml content.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The sources reference her his a different thing - I was referring to the Briefcase setting. Toga's sources are dynamic sources of data for lists, tables, trees etc.
|
|
||
| So, what if you want to generate a file through your app and save it? Toga provides four writable paths available for storing files associated with an app: | ||
|
|
||
| - ``data``: The location for storing user data. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we need to be explicit these are paths.data, etc?
| def load_button_pressed(self, button, **kwargs): | ||
| path = self.paths.config / "config.toml" | ||
| if not path.exists(): | ||
| path = self.paths.app / "resources/initial_config.toml" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Discrepancy here in "resources/initial_config.toml" vs "resources" / "initial_config.toml". Both are valid, but we should be consistent.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This was from the previous version. I had it updated in the actual Python file for this, but missed updating it here. Fixing.
Co-authored-by: Russell Keith-Magee <[email protected]>
Co-authored-by: Russell Keith-Magee <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've done a final review pass that has tweaked a few word/punctuation choice things, and some minor ordering tweaks in some case - but this is awesome. I've wanted this guide for a long time, and now it exists. Thanks for the contribution!
This is the current state of my first draft of the guide.
Issue:
PR Checklist: