shm_put_var(): Not enough shared memory left Error When Using SysVCacheItemPool with High Concurrency #597
Description
Environment details
- OS:
- Debian GNU/Linux 12 (bookworm)
- PHP version:
- 8.3.11
- Package name and version:
- google/cloud-spanner
- v1.84.0
- google/auth
- v1.42.0
- google/cloud-spanner
Hello,
I am using the google/cloud-spanner library to connect my PHP game server to Spanner. The code snippet below shows how I connect to Spanner:
new SpannerClient([
'authCache' => new SysVCacheItemPool(
[
'perm' => static::SHM_PERMISSION,
'memsize' => static::AUTH_SHM_MEM_SIZE,
],
),
'projectId' => $this->projectId,
'keyFilePath' => $this->keyFilePath,
]);However, when the number of users on the game server increases, the following error occurs in SysVCacheItemPool:
shm_put_var(): Not enough shared memory leftInvestigating the cause, it seems that when multiple accesses are made to PHP's shared memory with shm_put_var simultaneously, the error occurs. The following script reproduces the issue easily:
<?php
$shmid = shm_attach(1001, 1024 * 1024, 0666);
if (!$shmid) {
die("Failed to create shared memory segment\n");
}
$variableKey = 1;
while (true) {
$result = shm_put_var($shmid, $variableKey, ["only one"]);
if (!$result) {
echo "Failed to put variable into shared memory\n";
break;
}
}
shm_detach($shmid);Regarding the comment in SysVCacheItemPool:
This CacheItemPool implementation can be used among multiple processes, but it doesn't provide any locking mechanism. If multiple processes write to this ItemPool, you have to avoid race condition manually in your code.
I created a LockedSysVCacheItemPool that uses a locking mechanism (similar to the one used in CacheSessionPool) for writing to the shared memory, and this resolved the issue. Here is the code:
class LockedSysVCacheItemPool extends SysVCacheItemPool
{
use SysvTrait;
protected LockInterface $lock;
public function __construct($options = [])
{
parent::__construct($options);
$this->lock = $this->getDefaultLock($options);
}
/**
* Get lock object for synchronization
*
* @param array $options
*
* @return LockInterface
* @see CacheSessionPool::getDefaultLock()
*/
protected function getDefaultLock(array $options): LockInterface
{
$lockKey = sprintf(
'locked-sysv-cache-item-pool.%s.%s',
$options['proj'] ?? self::DEFAULT_PROJ,
$options['variableKey'] ?? self::VAR_KEY,
);
if ($this->isSysvIPCLoaded()) {
return new SemaphoreLock(
$this->getSysvKey(crc32($lockKey)),
);
}
return new FlockLock($lockKey);
}
/**
* {@inheritdoc}
*/
#[Override]
public function clear(): bool
{
return $this->lock->synchronize(function () {
return parent::clear();
});
}
/**
* {@inheritdoc}
*/
#[Override]
public function deleteItems(array $keys): bool
{
return $this->lock->synchronize(function () use ($keys) {
return parent::deleteItems($keys);
});
}
/**
* {@inheritdoc}
*/
#[Override]
public function save(CacheItemInterface $item): bool
{
return $this->lock->synchronize(function () use ($item) {
return parent::save($item);
});
}
}And here is the updated Spanner client configuration:
new SpannerClient([
'authCache' => new LockedSysVCacheItemPool(
[
'perm' => static::SHM_PERMISSION,
'memsize' => static::AUTH_SHM_MEM_SIZE,
],
),
'projectId' => $this->projectId,
'keyFilePath' => $this->keyFilePath,
]);This solution seems to have resolved the shm_put_var(): Not enough shared memory left error in my environment. I am reporting this here just in case others are facing the same issue.
Additionally, this may be related to the following issues:
#225
#226
If a pull request from me is necessary, I can create one.
Please note that since I am not fluent in English, I have used an AI to translate this message for me.