Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Tier0Feeder can crash on failed SM DB query #3976

@dballesteros7

Description

@dballesteros7

The same that happened to the RepackerAuditor. I guess the query needs to be wrapped in a except block to skip the error and try again next cycle.

012-09-25 13:09:21,650:INFO:Tier0FeederPoller:No workflows to publish to couch monitoring, doing nothing
2012-09-25 13:09:52,959:ERROR:BaseWorkerThread:Error in worker algorithm (1):
Backtrace:
  <T0Component.Tier0Feeder.Tier0FeederPoller.Tier0FeederPoller instance at 0x1c8ae950> (DatabaseError) ORA-01555: snapshot too old: rollback segment number 3 with name "_SYSSMU3_4024790241$" too small
 'SELECT a.runnumber, a.stream, a.lumisection, SUM(a.filecount)\n                 FROM CMS_STOMGR.streams a\n                 INNER JOIN CMS_STOMGR.runs b ON\n                   b.runnumber = a.runnumber AND\n                   b.instance = a.instance\n                 WHERE a.runnumber = :RUN\n                 AND a.stream = :STREAM\n                 AND a.lumisection > :LUMI\n                 GROUP BY a.runnumber, a.stream, a.lumisection\n                 HAVING ( COUNT(*) = ( SELECT COUNT(*)\n                                       FROM CMS_STOMGR.runs c\n                                       WHERE c.runnumber = a.runnumber ) AND\n                          COUNT(*) = ( SELECT MAX(c.n_instances)\n                                       FROM CMS_STOMGR.runs c\n                                       WHERE c.runnumber = a.runnumber ) )\n                 OR ( COUNT(*) = SUM(CASE\n                                       WHEN b.status = 1 THEN 1000\n                                       WHEN b.n_lumisections < a.lumisection THEN 0\n                                       ELSE 1\n                                     END) AND\n                      COUNT(*) = ( SELECT SUM(CASE\n                                                WHEN c.status = 1 THEN 1000\n                                                WHEN c.n_lumisections < a.lumisection THEN 0\n                                                ELSE 1\n                                              END)\n                                   FROM CMS_STOMGR.runs c\n                                   WHERE c.runnumber = a.runnumber ) )' {'RUN': 203584, 'STREAM': 'A', 'LUMI': 18}  File "/data/srv/wmagent/1.9.16/sw.pre/slc5_amd64_gcc461/cms/wmcore/0.9.13/lib/python2.6/site-packages/WMCore/WorkerThreads/BaseWorkerThread.py", line 160, in __call__
    self.algorithm(parameters)
  File "/data/srv/wmagent/1.9.16/sw.pre/slc5_amd64_gcc461/cms/t0/1.9.16/lib/python2.6/site-packages/T0Component/Tier0Feeder/Tier0FeederPoller.py", line 187, in algorithm
    RunLumiCloseoutAPI.closeLumiSections(self.dbInterfaceStorageManager)
  File "/data/srv/wmagent/1.9.16/sw.pre/slc5_amd64_gcc461/cms/t0/1.9.16/lib/python2.6/site-packages/T0/RunLumiCloseout/RunLumiCloseoutAPI.py", line 105, in closeLumiSections
    closedLumis = findClosedLumisDAO.execute(binds = runStreamLumis, transaction = False)
  File "/data/srv/wmagent/1.9.16/sw.pre/slc5_amd64_gcc461/cms/t0/1.9.16/lib/python2.6/site-packages/T0/WMBS/Oracle/RunLumiCloseout/FindClosedLumis.py", line 56, in execute
    transaction = transaction)[0].fetchall()
  File "/data/srv/wmagent/1.9.16/sw.pre/slc5_amd64_gcc461/cms/wmcore/0.9.13/lib/python2.6/site-packages/WMCore/Database/DBCore.py", line 167, in processData
    returnCursor = returnCursor))
  File "/data/srv/wmagent/1.9.16/sw.pre/slc5_amd64_gcc461/cms/wmcore/0.9.13/lib/python2.6/site-packages/WMCore/Database/DBCore.py", line 105, in executemanybinds
    resultproxy = connection.execute(s, bind)
  File "/data/srv/wmagent/1.9.16/sw.pre/slc5_amd64_gcc461/external/py2-sqlalchemy/0.5.2-comp8/lib/python2.6/site-packages/sqlalchemy/engine/base.py", line 824, in execute
    return Connection.executors[c](self, object, multiparams, params)
  File "/data/srv/wmagent/1.9.16/sw.pre/slc5_amd64_gcc461/external/py2-sqlalchemy/0.5.2-comp8/lib/python2.6/site-packages/sqlalchemy/engine/base.py", line 888, in _execute_text
    return self.__execute_context(context)
  File "/data/srv/wmagent/1.9.16/sw.pre/slc5_amd64_gcc461/external/py2-sqlalchemy/0.5.2-comp8/lib/python2.6/site-packages/sqlalchemy/engine/base.py", line 896, in __execute_context
    self._cursor_execute(context.cursor, context.statement, context.parameters[0], context=context)
  File "/data/srv/wmagent/1.9.16/sw.pre/slc5_amd64_gcc461/external/py2-sqlalchemy/0.5.2-comp8/lib/python2.6/site-packages/sqlalchemy/engine/base.py", line 950, in _cursor_execute
    self._handle_dbapi_exception(e, statement, parameters, cursor, context)
  File "/data/srv/wmagent/1.9.16/sw.pre/slc5_amd64_gcc461/external/py2-sqlalchemy/0.5.2-comp8/lib/python2.6/site-packages/sqlalchemy/engine/base.py", line 931, in _handle_dbapi_exception
    raise exc.DBAPIError.instance(statement, parameters, e, connection_invalidated=is_disconnect)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions