Updates to prepare_data function (openai#29)

* update documentation links to point to the website

* Fix encoding

* Add rough time estimator based on historical stats

* Fix train_test split naming logic; add quiet mode for running inside scripts

* Add a finetuning step by step example for a classification use case.

* add classification params if train and valid set; add length_validator

removing timeout and engine from args (openai#50) (openai#28)

renaming snapshot to model (openai#27)

* rename snapshot to model in sdk (openai#48)

* updating version

minor fixes to tools prepare_data validators (openai#47) (openai#26)

* ensure that only a single whitespace is prepended. Ensure the message regarding the prompt separator is displayed only if a prompt separator exists.

* change pandas contains to not use regex, which can trip if the common_suffix is actually a regex

Co-authored-by: Boris Power <[email protected]>

Cli fixes and improvements (openai#25)

* Revamp cli args (openai#45)

* Rachel/follow (openai#46)

* Add fine_tunes.follow. Add better error handling for disconnected streams

* return early

* fix an oops

* lint

* Nicer strings

* ensure end token is not applied to classification (openai#44)

* ensure end token is not applied to classification

* black

Co-authored-by: Boris Power <[email protected]>

bump version (openai#24)

bugfix

* ensure that pandas empty values are read as empty string, rather than a float

Co-authored-by: Boris Power <[email protected]>

Lots of CLI changes (openai#22)

* Add CLI option to download files (openai#34)

* Option to check if file has been uploaded in the past before uploading (openai#33)

The check is done based on filename, file purpose and file size

* Add fine-tuning hparams directly into the fine-tunes CLI (openai#35)

* update fine_tunes cli use_packing argument (openai#38)

* A file verification and remediation tool.

It applies the following validations:
- prints the number of examples, and warns if it's lower than 100
- ensures prompt and completion columns are present
- optionally removes any additional columns
- ensures all completions are non-empty
- infers which type of fine-tuning the data is most likely in (classification, conditional generation and open-ended generation)
- optionally removes duplicate rows
- infers the existence of a common suffix, and if there is none, suggests one for classification and conditional generation
- optionally prepends a space to each completion, to make tokenization better
- optionally splits into training and validation set for the classification use case
- optionally ensures there's an ending string for all completions
- optionally lowercases completions or prompts if more than a 1/3 of alphanumeric characters are upper case

It interactively asks the user to accept or reject recommendations. If the user is happy, then it saves the modified output file as a jsonl, which is ready for being used in fine-tuning with the printed command.

* Completion: remove  from kwargs before passing to EngineAPI (openai#37)

* Version bump before pushing to external

Co-authored-by: Todor Markov <[email protected]>
Co-authored-by: Boris Power <[email protected]>
Co-authored-by: Dave Cummings <[email protected]>

Support model-based (/engineless) completions in openai cli (openai#20)

* Support model-based (/engineless) completions in openai cli (openai#31)

* Engineless completions in the SDK

* Cosmetic improvements to fine tuning CLI

* Remove done TODO

* Raise error if neither engine nor model provided

* Undocument the `timeout` parameter on completions, because it doesn't do anything and causes user confusion.

* Move things around

* Update message

* Some day we should care about versions

* Minor version bump

Remove support for py <3.6, mypy, lots of cleanup (openai#19)

* python 3.6 + typing (openai#25)

- Formally require Python 3.6 or later. This never worked under Python 2.7.
- Drop now-unnecessary `__future__` and `six` references.
- Use `py.typed` to declare that we support type hints.
- Fix a variety of mypy errors.
- Don't use star imports, so that other packages can tell if they're making an invalid reference.
- Drop support for non-`requests` HTTP clients.
- Drop `EngineAPIResource.update`; it could never have worked.

Tested against primaryapi and engineapi in staging, and it doesn't break them. After this, `mypy ./openai` runs clean.

* delete unused code (openai#27)

- Delete `ListObject`. It has some typing errors that suggest it never worked, and it is unused as far as I can tell.
- Delete `VerifyMixin`. It is unused.
- Delete `CardError`. It is unused and smells of rotten pasta.
- Delete `OpenAIErrorWithParamCode`, which only has one subclass, `InvalidRequestError`, and make `InvalidRequestError` a direct subclass of `OpenAIError`. Currrently, `OpenAIErrorWithParamCode` depends on the internal structure of `InvalidRequestError` so they're not independent.

* boring formatting and typing fixes (openai#26)

These are another step towards being able to enforce black, flake8, and mypy on CI.

* fix more typing issues in prep for CI (openai#28)

- Simplify `platform.XXX` calls. As far as I know these can't raise an exception in Python 3.
- Simplify `EngineAPIResource` constructor and remove its unused `retrieve` method.

* Update readme, bump version

* typo fix

Co-authored-by: Madeleine Thompson <[email protected]>
Co-authored-by: Madeleine Thompson <[email protected]>