I'm concerned this is insufficient to make the store.DB threadsafe (which is I assume what you really want). For example, walking through two concurrent Do calls:

A: db.Open() // initializes db.lock and db.sqldb
B: db.Open() // also initializes db.lock and db.sqldb. Sort of no-op from the locking perspective, but now we leak two file handles, and...
A: db.DoTx(f) // ... now technically using B's sqldb/lock handles
A: db.Close() // deferred from earlier. Closes B's sqldb, and then sets db.sql to nil
B: db.DoTx() // panics trying to do `db.sql.Close`

Correct me if I'm totally misreading this, but otherwise there are several different race problems in the above.

@jonboulle #1892 (comment)
I don't understand why the B's Open() can pass the line lock.ExclusiveLock() before A closes the lock.

See this example:
https://gist.github.com/yifan-gu/50bfb8bffd839c219e7b

Happily, you're correct - I totally missed we're creating new file descriptors for each lock so then the flocks AREN'T reentrant.

Hm, I still wonder if there is some cleaner way we can beat the race detector though

Can we wrap the NewExclusiveLock with a store-specific mutex?

Can we wrap the NewExclusiveLock with a store-specific mutex?

Will do.

I figured out maybe it's the title that misleaded you, sorry :)

Actually I just put the mutex locking before the flock.

Updated

LGTM, but another pair of eyes would be good too.

@iaguis @alban @steveej ? Would require this as not to panic api service in rktnetes

or @vcaputo if he's around

@krnowak Thanks for the review :) Updated. Except for the ordering of unlocking.

lgtm, but are we sure this change is all that's needed to make safe what that panic and the existing lock implementation looks to be pretty intentionally preventing? @yifan-gu

@vcaputo So that previous panic assumes that there is no concurrent tries of acquiring the flock in one process (one single call of rkt cli). As now we provide a long running process which can handle requests concurrently, the assumption is not true anymore, so we should remove the panic. I achieved this by having the fd of the flock open during the whole life of the rkt process(api-service, run, fetch, etc.), so now it's safe for multiple concurrent threads to retry the flock.

The mutex doesn't have much effect here as flock also provides thread-safety, it's merely for the go race detector.

Makes sense

Small nitpicks, otherwise LFAD.

nit fixed. Will merge once this is green

Merging this to fix #1890