Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Camera API - Starting with esp32-camera #15753

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
cnadler86 opened this issue Aug 31, 2024 · 13 comments
Closed

Camera API - Starting with esp32-camera #15753

cnadler86 opened this issue Aug 31, 2024 · 13 comments
Labels
enhancement Feature requests, new feature implementations

Comments

@cnadler86
Copy link

cnadler86 commented Aug 31, 2024 •
edited
Loading

Description

As far as I searched the discussions and issues, I couldn't find any attempt to implement an interface to the esp32-camera driver in the oficial micropython repo. It should be not so difficult to implement these, since there are already some repos either having custom firmware with it, or also having an interface as source code (e.g. this one). I would take these as reference.

My propose would be the following:

  • Add the official esp32-camera driver from esp to the respective idf_component.yml as dependency.
  • Add a Compiler Flag to include the c interface module or not in the build, depending if the board definition (mpconfigboard.h) includes this flag,
  • In the board definition the respective default pins should be defined then,
  • The basic look and feel of the module in micropython for the user could be something like the actual implementations:
import camera
if camera.init(<keyword arguments>):
   img = camera.capture()
else:
   print('error')
  • Camera parameters would be get/set with camera methods.

Another possibility would be to handle the camera as python object. In this case properties could be used to get/set camera parameters. I guess in this case I would implement the camera as described above and build a frozen Micropython module on top.

What do you think?

Code Size

The code size would increase only in boards / variants that support camera from a HW-perspective (like this one: ESP32S3 XIAO SENSE)

Implementation

I intend to implement this feature and would submit a Pull Request if desirable, but I am happy on support from others

Code of Conduct

Yes, I agree
@cnadler86 cnadler86 added the enhancement Feature requests, new feature implementations label Aug 31, 2024
@dhalbert
Copy link
Contributor

We made an API for esp32-camera in CircuitPython, if that would serve for inspiration:
https://github.com/adafruit/circuitpython/tree/main/ports/espressif/bindings/espcamera
https://github.com/adafruit/circuitpython/tree/main/ports/espressif/common-hal/espcamera
https://docs.circuitpython.org/en/latest/shared-bindings/espcamera/index.html
We ended up forking it to make some small changes:
https://github.com/adafruit/esp32-camera

One issue that has come up is that the current version of esp32-camera uses the older, deprecated ESP-IDF I2C interface. We have been blocked moving to the newer I2C interface in general because of this (we are at ESP-IDF 5.3). However, someone just submitted a PR about this to esp32-camera: espressif/esp32-camera#677.

@cnadler86
Copy link
Author

cnadler86 commented Aug 31, 2024 via email

@cnadler86 cnadler86 mentioned this issue Aug 31, 2024
Open
@cnadler86
Copy link
Author

cnadler86 commented Sep 1, 2024 •
edited
Loading

I looked at the code posted from @dhalbert, and it looks very good. Thanks Dan one more time.
I think this API as basis would be the right approach to build the API for micropython, since it handles the camera interface as mpy-object. But of course is more sophisticated than the actual implementations for micropython.

I would not fork the esp32-camera driver (since the i2c interface will be updated soon) and use the one provided by espressif, because I and probably also the mpy-community would not have time or want to maintain this fork.

The questions would be: do we want to have an camera API in Micropython? And if yes, do we want to have the same interface to the outside as circuitython, making camera code compatible between micropython and circuitpython, or do we want to define a own micropython interface (thinking forward: the camera API should have the same look and feel for users in other Ports). AFAIK there are no camera APIs in other micropython ports, or? Probably i should now change the name of the issues to something like "camera API, starting with esp32-camera" 🤪

@jonnor
Copy link
Contributor

jonnor commented Sep 1, 2024

OpenMV also has an API for cameras, going back many years. https://docs.openmv.io/library/omv.sensor.html

@cnadler86
Copy link
Author

OpenMV also has an API for cameras, going back many years. https://docs.openmv.io/library/omv.sensor.html

Didn't know this. Thanks for pointing it out.

What I see now is that there are at least 2 more or less big stakeholders (Adafruit and OpenMV), maybe others having their own camara interface based on micropython. So, IMHO the right way of implementing a camera interface (with a well defined HAL, Port-APIs, etc.) that could suit the demands of these stakeholders would be to start a sponsored project for that, so these stakeholder and everyone else using micropython as basis could use it. But it seams that this train has already left the station...

BTW: In these case I am for sure not the right guy to implement this. I (naively) thought I would start more or less from a green field with some examples and build an API for hobbyists, but it seams that this might not be the case.

@dhalbert
Copy link
Contributor

dhalbert commented Sep 2, 2024

Maybe I am misinterpreting what you mean by "stakeholder". I would not call us, Adafruit, a stakeholder. We have no demands: MicroPython can have whatever camera API it wants. I was just suggesting that our implementation might be helpful to you as inspiration for the API and as an example of code that works that you could take or adapt. By the same measure, OpenMV is not a stakeholder, but can serve as similar inspiration.

@cnadler86
Copy link
Author

cnadler86 commented Sep 2, 2024 •
edited
Loading

No, I think I formulated my thought the wrong way: Since micropython is the source of circuitpython, OpenMV and probably others, I think the biggest benefit for all developers using micropython together with a camera would be to have such an API that suits all their needs in the source (of course this is utopic, but that would be my vision). Since the source didn't have such an API, these groups developed/reused their own. Implementing now a Camera-API can have advantages for the groups that already have an API (of course only if they switch to it), such as maintaining the API by a wider community, finding bugs then probably faster, etc., but also having less flexibility when it comes to customization.
And of course implementing an API in MP is now much easier, because we can learn from the camera-API-journey of this groups =).
Perhaps what I am trying to say or put to discussion: Are these groups interested in migrating their APIs to MP and how active want they to get involved in it?

At the moment my plan would be more or less the following:

  • implementing the esp32-cam driver with the respective additions in probably as machine module for modmachine + bindings
  • implementing the higher level Camera-API, based on these ESP-low level SW and the already available API-implementations
  • all these for the basic functionality with a basic validation, trying to have an acceptable performance and memory usage, tested in 1-2 camera sensors (I cannot offer more than that).

@cnadler86
Copy link
Author

cnadler86 commented Sep 11, 2024 •
edited
Loading

So, I have a first alpha, which I could compile and successfuly test the basics on my esp32s3 with the ov2640. This is of course a work in progress and the information, comments, etc. might not be good, but it is a starting point.

Here the link

Note: if you want to use it as it is now, you will need to hard code the configuration in a constructor stub or debug the actual constructor 🥴

@jonnor
Copy link
Contributor

jonnor commented Sep 13, 2024

Very cool @cnadler86 !

@cnadler86
Copy link
Author

cnadler86 commented Sep 17, 2024 •
edited
Loading

Hello strangers,

Have finally a release candidate which I would call stable and OK-documented. Feel free to use it or comment on it.

I still think it would be good to incorporate this in micropython, but if this is not the strategy or the wish of the community, at least I would suggest to put it in awesome-micropython and leave it as user-module.

Thanks once again @dhalbert and @jonnor for providing the key informations for the mpy-implementation.

@cnadler86 cnadler86 changed the title ESP32-camera support Camera API - Starting with esp32-camera Sep 18, 2024
@jonnor
Copy link
Contributor

jonnor commented Sep 25, 2024

Definitely get this into awesome-micropython. To do that, the best is to submit a pull request following the contributions guidelines: https://github.com/mcauser/awesome-micropython/blob/master/contributing.md#submitting-pull-requests
I have found the maintainer to be very responsive and friendly :)

I actually realize I have an ESP32+OV2640 device - maybe I will test it out on that device.

@jonnor
Copy link
Contributor

jonnor commented Sep 25, 2024

Having an issue when testing on ESP32 - iram overflow. I filed an issue in the repo, cnadler86/micropython-camera-API#1

@cnadler86
Copy link
Author

Hi,

since here is not happening much I will close the Issue. The API is now in awesome-micropython. I will let the topic mature and depending on the resonance, open a PR to merge it in here in the future.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement Feature requests, new feature implementations
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@jonnor @dhalbert @cnadler86