adding per-lambda limits #329
Conversation
…limits merge: update feature branch with latest main after cleanup
| @@ -23,22 +23,13 @@ func SandboxPoolFromConfig(name string, sizeMb int) (cf SandboxPool, err error) | |||
| return nil, fmt.Errorf("invalid sandbox type: '%s'", common.Conf.Sandbox) | |||
| } | |||
|
|
|||
| func fillMetaDefaults(meta *SandboxMeta) *SandboxMeta { | |||
There was a problem hiding this comment.
Why get rid of this? Isn't it more convenient to populate everything in meta instead of having a separate memLimitMB in SOCKContainer? I don't want to add a bunch of fields to SOCKContainer as we have more resource types.
There was a problem hiding this comment.
I removed fillMetaDefaults because I wanted the resource defaults to be resolved closer to sandbox creation, and thought it might be cleaner to pass everything through Limits. That said, I see your concern about duplicating fields in SOCKContainer. I can reintroduce fillMetaDefaults and avoid the extra field so we don’t scatter defaults across places.
src/worker/sandbox/api.go
Outdated
| CPUPercent int | ||
| Installs []string | ||
| Imports []string | ||
| Limits *common.Limits // Optional resource limits to override worker defaults. |
There was a problem hiding this comment.
I think things we be cleaner if we make it not optional.
There was a problem hiding this comment.
The reason I kept it optional was to allow distinguishing between “use worker defaults” vs “explicitly set to 0.” But I agree it adds complexity, and it should be cleaner if we always populate Limits. I’ll update this so Limits is non-optional and always available.
| } | ||
|
|
||
| t2 := common.T0("ImportCache.createChildSandboxFromNode:childSandboxPool.Create") | ||
| // Use the meta object from the function invocation, which contains the correct limits. |
There was a problem hiding this comment.
Not sure we need the comment. Is it correct? Or could the child sandbox we're creating be another zygote, not associated with the current function request?
There was a problem hiding this comment.
You’re right - the comment isn’t entirely accurate because the child sandbox could itself be a zygote.
src/worker/lambda/lambdaInstance.go
Outdated
| maxRuntime = time.Duration(linst.lfunc.Meta.Config.MaxRuntimeSec) * time.Second | ||
| } | ||
|
|
||
| // make the request in a goroutine |
There was a problem hiding this comment.
Do we need to create a new goroutine for this? Client already has a Timeout field https://pkg.go.dev/net/http#Client. Timeout is probably something we need to look at more closely, but as a separate PR. I think we should look at Go contexts and see if they can help us do timeout more elegantly (https://pkg.go.dev/context#example-WithTimeout).
There was a problem hiding this comment.
Good point. I added the goroutine approach because I wanted per-lambda runtime limits independent of the global client timeout. But I agree that Client.Timeout or contexts would be a better long-term solution. Happy to drop the goroutine for now and explore contexts in a follow-up PR.
| sb, err = f.lmgr.ZygoteProvider.Create(f.lmgr.sbPool, true, linst.codeDir, scratchDir, meta, f.rtType) | ||
| if err != nil { | ||
| f.printf("failed to get Sandbox from import cache") | ||
| f.printf("failed to get Sandbox from import cache: %v", err) |
There was a problem hiding this comment.
Makes sense - I’ll switch to %w for proper error wrapping.
src/worker/lambda/lambdaFunction.go
Outdated
| pkg = packages.NormalizePkg(pkg) | ||
| sandboxMeta.Installs = append(sandboxMeta.Installs, pkg) | ||
|
|
||
| if file != nil { |
There was a problem hiding this comment.
Perhaps we can move some of this to a separate parseRequirementsTxt to keep parseMeta simpler?
There was a problem hiding this comment.
Yep, I’ll refactor requirements parsing into a parseRequirementsTxt helper so parseMeta stays focused.
src/common/lambdaConfig.go
Outdated
| Triggers Triggers `yaml:"triggers"` // List of HTTP triggers | ||
| // Additional configurations can be added here. | ||
| Limits Limits `yaml:"limits,omitempty"` | ||
| MaxRuntimeSec int `yaml:"max_runtime_sec,omitempty"` |
There was a problem hiding this comment.
I wonder if MaxRuntimeSec should be part of Limits?
There was a problem hiding this comment.
I put it in LambdaConfig separately since it felt more like a behavior toggle than a resource limit. I’m open to moving it under Limits to keep configs consistent!
83f263c to
3ac5525
Compare
…y and include RuntimeSec
…y and include RuntimeSec
tylerharter
left a comment
There was a problem hiding this comment.
Emailed some thoughts, but most important is that this should be multiple, smaller PRs. E.g., all the improved error message stuff should be separate.
No description provided.