Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@alexandruradovici
Copy link
Contributor

Pull Request Overview

This pull request adds:

  1. A HIL proposal for frame buffer support. This will allow applications to use graphical LCD and EPD screens. The longer purpose is to enable littlevgl.
  2. The frame buffer capsule to be used by the user applications
  3. A capsule to support the ST7735 SPI screen.

The frame buffer implementation allows zero copy mechanism for writing data from the user application to the screen memory. This is done by using a callback function to fill a screen buffer from an AppSlice.

pub trait ScreenClient {
    // this function is called by the screen whenever it has to send data
    // it returns the number of bytes placed in the buffer or 
    // 0 if there are no more bytes to write
    fn fill_next_buffer_for_write(&self, buffer: &'a mut [u8]) -> usize;
    fn command_complete(&self, r: ReturnCode);
}

In the screen implementation, this function is used like this:

self.buffer.take().map_or_else(
    || panic!("st7735: send parameters has no buffer"),
    |buffer| {
        let len = client.fill_next_buffer_for_write(buffer);
        if len > 0 {
            self.status.set(Status::SendCommandSlice(len));
            self.dc.set();
            self.spi.read_write_bytes(buffer, None, len);
        } else {
            self.buffer.replace(buffer);
            self.do_next_op();
        }
    },
);

Testing Strategy

This pull request was tested with a nucleo429zi and an Adafruit ST7735 screen.

TODO or Help Wanted

This pull request still needs some feedback regarding the frame buffer API,. Any suggestion is welcome.

Documentation Updated

  • Updated the relevant files in /docs, or no updates are required.

Formatting

  • Ran make formatall.

@alexandruradovici alexandruradovici changed the title Framebuffer HIL and ST7735 screen support Frame buffer HIL and ST7735 screen support May 8, 2020
Copy link
Contributor

@bradjc bradjc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some HIL questions to get things started.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is invert something that is generally supported by displays?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am still waiting for an e-ink display, but I think so. I'll get back on that.

Copy link
Member

@ppannuto ppannuto left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm lightly concerned about all of these interfaces being synchronous. I could imagine a display connected over I2C or similar bus with a display controller on the other end that could never implement all of these synchronously. Maybe the overhead of async is too much for the local screen case though -- we certainly have the pattern of a sync and async version already from the GPIO HILs for this reason (i.e. a GPIO extender must be async, but making GPIO async in the common case would be obnoxious)

@alexandruradovici
Copy link
Contributor Author

I'm lightly concerned about all of these interfaces being synchronous. I could imagine a display connected over I2C or similar bus with a display controller on the other end that could never implement all of these synchronously. Maybe the overhead of async is too much for the local screen case though -- we certainly have the pattern of a sync and async version already from the GPIO HILs for this reason (i.e. a GPIO extender must be async, but making GPIO async in the common case would be obnoxious)

All the set functions are asynchronous as they need to interact with the display. The get functions are not, as the driver will have to configure the display to a certain resolution and pixel format, and I thought that the driver can memorize these options. Is there any way that the display could have a resolution and pixel format unknown to the driver?

Copy link
Contributor

@bradjc bradjc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems like a great candidate to be split into basic and advanced HILs. The basic HIL would just support the get_ commands and write, and the current HIL would be come the advanced version. The impacts on the other code in this PR would be minimal: there would just be a little renaming. I think the split would be a little nicer than just having an implementation for a simple display return errors for most functions, and making the upper layers handle that.


I think some more clarity around how pixel values should be packed is needed. I'm not sure if this should use a type of some sort, or more comments, or just a comment saying it is up to specific implementations.


The other bit is the .write() structure, which is an inverse to how all other operations in Tock work. Namely: the buffer is owned by the lower layer, and is passed up in a callback when requested, filled by the upper layer, and then used when the callback returns.

I'm actually not really sure why this works. How does the lower layer know it will have access to the buffer once fill_next_buffer_for_write() returns? What if the upper layer needs to wait for more data from an app, and wants to save the reference to the buffer?

In any case, it would be helpful to articulate and document why this doesn't follow the normal pattern of buffers being owned by the upper layers and passed to the lower layers. I'm not sure I see why the buffer couldn't be passed in with write(), and returned in the callback.

Am I correct in assuming that display hardware often keeps track of where the user is writing to, and just expects more data to be fed until all pixels in that area have been set? If so, I think it makes sense to capture that in the HIL. Adding a continue_write(self, buf) function would allow for this.

@alexandruradovici
Copy link
Contributor Author

This seems like a great candidate to be split into basic and advanced HILs. The basic HIL would just support the get_ commands and write, and the current HIL would be come the advanced version. The impacts on the other code in this PR would be minimal: there would just be a little renaming. I think the split would be a little nicer than just having an implementation for a simple display return errors for most functions, and making the upper layers handle that.

I'll change it this way, it makes sense.

I think some more clarity around how pixel values should be packed is needed. I'm not sure if this should use a type of some sort, or more comments, or just a comment saying it is up to specific implementations.

These seems to be pretty standard ways of encoding the pixels. I added comments to each of them and most of the screen / ui libraries support them (ex: lvgl).

The other bit is the .write() structure, which is an inverse to how all other operations in Tock work. Namely: the buffer is owned by the lower layer, and is passed up in a callback when requested, filled by the upper layer, and then used when the callback returns.

I'm actually not really sure why this works. How does the lower layer know it will have access to the buffer once fill_next_buffer_for_write() returns? What if the upper layer needs to wait for more data from an app, and wants to save the reference to the buffer?

The lifetime for the buffer is related to the function's scope, not to the trait scope. The function should not be able to keep the buffer. I may be wrong as I do not have a lot of experience with reference lifetimes.

fn fill_next_buffer_for_write(&self, buffer: &'b mut [u8]) -> usize;

In any case, it would be helpful to articulate and document why this doesn't follow the normal pattern of buffers being owned by the upper layers and passed to the lower layers. I'm not sure I see why the buffer couldn't be passed in with write(), and returned in the callback.

Am I correct in assuming that display hardware often keeps track of where the user is writing to, and just expects more data to be fed until all pixels in that area have been set? If so, I think it makes sense to capture that in the HIL. Adding a continue_write(self, buf) function would allow for this.

I chose this implementation in order to minimize the number of buffer copies required. Screen buffers can be large, so I wanted a way to copy data directly from the AppSlice to the screen. I haven't found a way to pass the AppSlice down (probably there is no safe way).

The screen driver has a buffer for sending data over to the screen (in this example an SPI buffer). The buffer size varies depending on the screen's implementation and bus type. As the frame buffer has no knowledge about this, I thought that the best way would be for the screen to ask for an amount of data from the frame buffer before performing any write.

If the buffer is passed down, I still need a small buffer for other command writes inside the screen and the buffer passed down from the frame buffer, in other words both capsules (framebuffer and screen) need a static buffer.

I think I could pass the buffer down using continue_write ... let me try an implementation like that.

@bradjc bradjc added the HIL This affects a Tock HIL interface. label May 18, 2020
@alexandruradovici
Copy link
Contributor Author

I made the following changes to the HIL so that the buffer is sent from the frame buffer driver towards the lower level screen driver:

/// Sets the video memory frame.
fn set_write_frame(&self, x: usize, y: usize, width: usize, height: usize) -> ReturnCode;

/// Sends a write command to write data in the selected video memory frame.
fn write(&self, buffer: &'static mut [u8], len: usize) -> ReturnCode;

I added another function to the ScreenClient to signal when a write command is complete.

/// The screen will call this function to notify that a command (except write) has finished.
fn command_complete(&self, r: ReturnCode);

/// The screen will call this function to notify that the write command has finished.
/// This is different from `command_complete` as it has to pass back the write buffer
fn write_complete(&self, buffer: &'static mut [u8], r: ReturnCode);

Any feedback is very useful.

@alexandruradovici
Copy link
Contributor Author

Any ideas why the check fails? It seems some timeout on opentitan, but I see no error.

@hudson-ayers
Copy link
Contributor

I re-ran it and it passed -- it seems the QEMU test is not completely deterministic at this point (cc @alistair23 ).

@alexandruradovici
Copy link
Contributor Author

I re-ran it and it passed -- it seems the QEMU test is not completely deterministic at this point (cc @alistair23 ).

Thank you

@alistair23
Copy link
Contributor

I can't see what failed, but if it was Travis then that is probably related to cache.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I understand how this would be used. I imagine it would be easy to implement, but what can clients expect to do with the number of supported resolutions?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some screens like ST7735 seem to support several resolution, I thought the user might want to select one.

bradjc
bradjc previously approved these changes Jun 16, 2020
@bradjc bradjc added the last-call Final review period for a pull request. label Jun 16, 2020
ppannuto
ppannuto previously approved these changes Jun 16, 2020
Copy link
Member

@ppannuto ppannuto left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is fantastic, thank you.

There's a bunch of smaller formatting kind of things I noticed doing a quick final pass, but nothing that I would block this on.

@alexandruradovici alexandruradovici dismissed stale reviews from ppannuto and bradjc via 65174ad June 17, 2020 21:53
@bradjc
Copy link
Contributor

bradjc commented Jun 18, 2020

Why does this delete an I2C driver?

@alexandruradovici
Copy link
Contributor Author

Why does this delete an I2C driver?

The file was not supposed to be there, the I2C is in stm32f303xc, the deleted file was left there by mistake.

@bradjc
Copy link
Contributor

bradjc commented Jun 18, 2020

bors r+

@bors
Copy link
Contributor

bors bot commented Jun 18, 2020

@bors bors bot merged commit 313a5e1 into tock:master Jun 18, 2020
bors bot added a commit to tock/libtock-c that referenced this pull request Jun 29, 2020
86: Frame buffer API for user space and lvgl port r=ppannuto a=alexandruradovici

This pull request:
1. adds the frame buffer API for tock/tock#1837.
2. adds a simple driver for lvgl (https://github.com/littlevgl/lvgl)


Co-authored-by: Alexandru Radovici <[email protected]>
This was referenced Apr 4, 2022
tyler-potyondy pushed a commit to tyler-potyondy/libtock-c that referenced this pull request Mar 13, 2024
86: Frame buffer API for user space and lvgl port r=ppannuto a=alexandruradovici

This pull request:
1. adds the frame buffer API for tock/tock#1837.
2. adds a simple driver for lvgl (https://github.com/littlevgl/lvgl)


Co-authored-by: Alexandru Radovici <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

HIL This affects a Tock HIL interface. last-call Final review period for a pull request.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants