Thanks to visit codestin.com
Credit goes to github.com

Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
121 commits
Select commit Hold shift + click to select a range
72990ba
[fetcher] getLastSeenAccountState: do not count states
iam047801 May 17, 2024
8a3685c
[indexer] getMessageSource: fix panic message
iam047801 May 18, 2024
1e3ab2d
[parser] fix error wrappings on failed GetMethodDescription
iam047801 May 18, 2024
ba3690a
docker-compose.yml: restart services on failure
iam047801 May 18, 2024
6b3c215
[parser] add semaphore to limit maximum number of account parsing wor…
iam047801 May 18, 2024
7353f92
[repo] contract: do not remove standard minter get-method descriptions
iam047801 May 18, 2024
800906b
[fetcher] getOtherAccount: add time track
iam047801 May 18, 2024
a05ec13
[parser] add lru cache for minter addresses
iam047801 May 18, 2024
f55791b
[fetcher] getOtherAccount: fetch minter state exactly once
iam047801 May 18, 2024
7fce15c
[repo] account filter: always limit number of rows
iam047801 May 18, 2024
d468ad7
[fetcher] getAccount: fetch account state on a block exactly once
iam047801 May 18, 2024
8cd2e2b
[parser] add some time tracking
iam047801 May 18, 2024
ee35e25
[repo] contract: add mutex for GetInterfaces
iam047801 May 18, 2024
81109e5
[repo] add time track for interfaceMatched
iam047801 May 19, 2024
ef83dd3
[repo] contract: fill and cache contract code hash
iam047801 May 19, 2024
ca8d59c
[parser] emulateGetMethod: add time track
iam047801 May 19, 2024
abbe03f
[repo] account: add timer for getCodeData
iam047801 May 19, 2024
fa19cfc
[indexer] getMessageSource: fix check for masterchain messages
iam047801 May 19, 2024
497785e
[indexer] fetch and save blocks simultaneously
iam047801 May 19, 2024
605033a
[indexer] publishProcessedBlocks: fix typo
iam047801 May 19, 2024
ea6c3ac
[fetcher] getAccount: rewrite account cache in case of error
iam047801 May 21, 2024
b1dedec
[indexer] fetchMastersConcurrent: catch up with maximum number of wor…
iam047801 May 29, 2024
c452f5b
[fetcher] getAccount: start timer after cache get
iam047801 May 30, 2024
a189a17
[fetcher] getAccountLibraries: add timer
iam047801 May 30, 2024
69969b5
[repo] tx: do not ch count rows on filter by hash
iam047801 Jun 19, 2024
18fb161
SkipAddress: add quackquack account states
iam047801 Jun 21, 2024
d47585b
[fetcher] fix tests
iam047801 Jun 21, 2024
2806da5
[cmd] add fillMissedClickHouseData command
iam047801 Jul 24, 2024
4e8b3e3
[cmd] fillMissedClickHouseData: check data length on insert
iam047801 Jul 24, 2024
b61cc48
[cmd] fillMissedClickHouseData: check incoming message is not nil
iam047801 Jul 24, 2024
5ddd62f
[cmd] fillMissedClickHouseData: fix implicit memory aliasing
iam047801 Jul 24, 2024
6e75921
[indexer] fetchMastersConcurrent: fix ineffectual assignment to workers
iam047801 Jul 24, 2024
e3f3a67
[repo] contract: fix interface tests
iam047801 Jul 24, 2024
5adfa47
[repo] do not count total number of rows by default
iam047801 Jul 24, 2024
c4998fb
[api] do not count total number of rows by default
iam047801 Jul 24, 2024
f1ac9c1
[api] transactions filter: total rows omitempty
iam047801 Jul 24, 2024
647aef0
[cmd] db: add subcommand to fill missed raw account states code and data
iam047801 Aug 4, 2024
742d2d7
[cmd] db.fillMissedRawAccountStates: fix typo on account states select
iam047801 Aug 4, 2024
b86c03e
[cmd] db.fillMissedRawAccountStates: check code and data hashes are n…
iam047801 Aug 4, 2024
955e58c
[cmd] db.fillMissedRawAccountStates: add log for number of account st…
iam047801 Aug 4, 2024
f37ca0f
[cmd] db.fillMissedRawAccountStates: get code and data in batches
iam047801 Aug 4, 2024
5447988
[cmd] db.fillMissedRawAccountStates: stop when number of scanned acco…
iam047801 Aug 5, 2024
2014dcd
SkipAddress: add some mainnet addresses
iam047801 Aug 23, 2024
4b241b0
SkipAddress: add heavy mainnet highload v2 wallet
iam047801 Aug 23, 2024
dfd9a4d
SkipAddress: add heavy mainnet nft collection and jetton distribution…
iam047801 Aug 23, 2024
2c28256
SkipAddress: add eth token bridge collector and some other unknown he…
iam047801 Aug 23, 2024
5c0c6ca
SkipAddress: one more jetton token distribution address
iam047801 Aug 23, 2024
2bd964d
[repo] AddMessages: batch insert
iam047801 Aug 29, 2024
a722096
[indexer] insert blocks in batches
iam047801 Aug 29, 2024
25261da
[indexer] saveBlocks: add context with timeout
iam047801 Aug 29, 2024
4b0cc2d
[repo] ConnectDB: add write timeout for pg
iam047801 Aug 29, 2024
888b39e
[indexer] insertData: context with timeout before inserts
iam047801 Aug 29, 2024
bebf1bc
[indexer] uniqMessages: add timer
iam047801 Aug 29, 2024
4c59abc
[indexer] refactor uniqMessages to optimize message source check
iam047801 Aug 29, 2024
e891739
[indexer] publishProcessedBlocks: fix slice copying
iam047801 Sep 30, 2024
7d3186b
[query] GetStatistics: fix method to only one thread, add cache
iam047801 Oct 4, 2024
677921d
[core] SkipAddress: add gemz, wonton
iam047801 Oct 26, 2024
35811ab
Dockerfile: apt-get install autoconf libtool
iam047801 Dec 25, 2024
b260164
[core] SkipAddress: add more addresses
iam047801 Jan 13, 2025
4949633
[query] calculate statistics in background
iam047801 Mar 1, 2025
90259b3
[cmd] query: start update stats loop
iam047801 Mar 1, 2025
ca5a6f7
[repo] message: add cache for messages count
iam047801 Jun 20, 2025
93f24dd
[repo] account: simplify states counting on filter
iam047801 Jun 20, 2025
d438121
[core] write to the new table for latest parsed account states
iam047801 Jun 20, 2025
8f7704e
[migrations] fix latest parsed account states migration
iam047801 Jun 20, 2025
62cb136
[core] remove latest_parsed_account_states table, write to latest_acc…
iam047801 Jun 21, 2025
2162cf2
[migrations] fix latest_account_states procedure for table population
iam047801 Jun 21, 2025
1549ace
[repo] filterAccountStates: use latest_account_states for types and o…
iam047801 Jun 21, 2025
87c99c4
[api] internalErr: print url on error
iam047801 Jun 21, 2025
a1b2420
[api] handle context cancellation properly
iam047801 Jun 21, 2025
3f0d63c
[core] latest_account_states: add created_lt
iam047801 Jun 21, 2025
79339f7
[migrations] comment out procedures
iam047801 Jun 21, 2025
8dd969b
[migrations] account_states created_lt: fix procedure
iam047801 Jun 21, 2025
40ca1b1
[repo] countMsgFullScan: handle empty messages table and unfiltered m…
iam047801 Jun 21, 2025
8edec35
[repo] countMsgPartialScan: unfiltered max created lt and handle case…
iam047801 Jun 21, 2025
e300b31
[repo] account filter: cache counting, do full scan on types/owner/mi…
iam047801 Jun 21, 2025
45ce9d3
[repo] countAccountStates / countMsg: handle full scan not found erro…
iam047801 Jun 21, 2025
e66ab92
[repo] transaction: cache counting
iam047801 Jun 21, 2025
b1f50bd
[core] convert SkipAddress function to SkippedAddresses map
iam047801 Jun 22, 2025
6aa631d
[fetcher] simplify getAccount
iam047801 Jun 22, 2025
c9825d4
some simple linter fixes
iam047801 Jun 22, 2025
acdea88
remove nolint directives
iam047801 Jun 22, 2025
cbdca81
[migration] add db transactions
iam047801 Jun 22, 2025
c2f547f
[repo] tune filter cache ttl
iam047801 Jun 22, 2025
1d6db34
[repo] tune messages filter cache ttl
iam047801 Jun 22, 2025
900653e
[repo] messages filter counting: round created_lt for cache
iam047801 Jun 22, 2025
c68fef7
[repo] countMsgPartialScan: fix query
iam047801 Jun 22, 2025
cc59ec6
[migrations] fix batch_fill_account_states_created_lt procedure
iam047801 Jun 23, 2025
8885e84
upgrade tonutils-go to v1.13.0
iam047801 Jun 23, 2025
509777b
github actions: update go version for golangci-lint
iam047801 Jun 23, 2025
110bc33
golangci-lint: migrate to v2.1
iam047801 Jun 23, 2025
ff4a2de
[repo] message: only 1 hour cache
iam047801 Jun 23, 2025
d75ca38
.golangci.yaml: remove depguard
iam047801 Jun 23, 2025
9633147
go.mod: go version 1.23.0
iam047801 Jun 23, 2025
88bca32
[migrations] fix comments
iam047801 Jun 23, 2025
4af29b6
[migrations] latest_account_states created_lt as not null
iam047801 Jun 23, 2025
956cc51
[fetcher] getAccountLibraries: skip nil libraries
iam047801 Jun 24, 2025
fa0a123
[query] FilterAccounts: validate contract types
iam047801 Jun 24, 2025
ab5bcd9
[query] AggregateAccountsHistory: validate contract types
iam047801 Jun 24, 2025
f496a0b
[query] FilterMessages/AggregateMessagesHistory: validate contract ty…
iam047801 Jun 24, 2025
b28ec85
[abi] move emulator to a separate package
iam047801 Jun 24, 2025
ec148bc
README.md: provide emulator library path to run tests
iam047801 Jun 24, 2025
4d6db49
[filter] cache: cleanup old entries on Get
iam047801 Jun 24, 2025
203c890
[statistics] getTransactionStatistics: do not panic if there are no m…
iam047801 Jun 25, 2025
f34d06e
[indexer] getMessagesSource: fail on unknown message source after 16 …
iam047801 Jun 25, 2025
7970fa5
[fetcher] mapMessage: calculate message hash as in old versions of to…
iam047801 Jun 25, 2025
89d8d3b
[repo] message filter hash: lag for rounded max_lt
iam047801 Jun 25, 2025
8e6a81f
.golangci.yaml: do not check msg_hash folder
iam047801 Jun 25, 2025
625bc80
[rndm] unify lt counter for tests
iam047801 Jun 25, 2025
4b59dd8
[repo] message filter counting: fix tests and since_start_count in pa…
iam047801 Jun 25, 2025
9cbc6df
[repo] countAccountStates: use rounded counting
iam047801 Jun 25, 2025
c451cf2
[repo] account filter counting: remove countLatestAccountStatesFullSc…
iam047801 Jun 25, 2025
7c4bc66
[repo] countAccountStatesFullScan: remove filter check
iam047801 Jun 25, 2025
caa6129
[repo] countAccountStates: full scan postgresql instead of clickhouse
iam047801 Jun 25, 2025
7f74263
[repo] tx filter count: round max lt
iam047801 Jun 26, 2025
5a2bce5
[repo] countAccountStates: nolint nestif
iam047801 Jun 26, 2025
9f770f9
Merge pull request #46 from tonindexer/optimize-count
iam047801 Jun 26, 2025
b498510
[core] latest account states: add fake flag
iam047801 Jun 26, 2025
d649719
[repo] AggregateAccounts: move queries to postgresql
iam047801 Jun 26, 2025
bd505af
README.md: prettify cgo lib export example
iam047801 Jun 27, 2025
d7db10e
[migrations] latest_account_states: drop fake column in down migration
iam047801 Jun 27, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 5 additions & 6 deletions .github/workflows/golangci-lint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,12 +19,11 @@ jobs:
name: lint
runs-on: ubuntu-latest
steps:
- uses: actions/setup-go@v4
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version: '1.19'
cache: false
- uses: actions/checkout@v3
go-version: stable
- name: golangci-lint
uses: golangci/golangci-lint-action@v3
uses: golangci/golangci-lint-action@v8
with:
version: v1.52.2
version: v2.1
124 changes: 69 additions & 55 deletions .golangci.yaml
Original file line number Diff line number Diff line change
@@ -1,46 +1,36 @@
version: "2"
run:
go: '1.19'
concurrency: 4
timeout: 5m
go: "1.23"
modules-download-mode: readonly
issues-exit-code: 2
tests: true
modules-download-mode: readonly
allow-parallel-runners: false
skip-files:
- main.go
linters:
disable-all: true
default: none
enable:
- errcheck
- gosimple
- govet
- ineffassign
- staticcheck
- unused
- asciicheck
- asciicheck
- bidichk
- decorder
- depguard
# - depguard
- dupl
- durationcheck
- errcheck
- errchkjson
- errname
- errorlint
- execinquery
- exportloopref
- forbidigo
- forcetypeassert
- goimports
- gocognit
- goconst
- gocritic
- gocyclo
- gofmt
- goheader
- gosec
- govet
- grouper
- importas
- ineffassign
- ireturn
- maintidx
- makezero
Expand All @@ -51,42 +41,66 @@ linters:
- nolintlint
- predeclared
- promlinter
- staticcheck
- unconvert
- unused
- whitespace
linters-settings:
gocyclo:
min-complexity: 18
gosec:
excludes:
- G404
gocritic:
disabled-checks:
- regexpMust
- commentedOutCode
- docStub
enabled-tags:
- diagnostic
- style
- performance
- experimental
- opinionated
settings:
captLocal:
paramsOnly: false
elseif:
skipBalanced: false
nestingReduce:
bodyWidth: 4
rangeValCopy:
sizeThreshold: 64
skipTestFuncs: false
tooManyResultsChecker:
maxResults: 100
truncateCmp:
skipArchDependent: false
underef:
skipRecvDeref: false
unnamedResult:
checkExported: true
hugeParam:
sizeThreshold: 64
settings:
gocritic:
disabled-checks:
- regexpMust
- commentedOutCode
- docStub
enabled-tags:
- diagnostic
- style
- performance
- experimental
- opinionated
settings:
captLocal:
paramsOnly: false
elseif:
skipBalanced: false
hugeParam:
sizeThreshold: 64
nestingReduce:
bodyWidth: 4
rangeValCopy:
sizeThreshold: 64
skipTestFuncs: false
tooManyResultsChecker:
maxResults: 100
truncateCmp:
skipArchDependent: false
underef:
skipRecvDeref: false
unnamedResult:
checkExported: true
gocyclo:
min-complexity: 18
gosec:
excludes:
- G404
exclusions:
generated: lax
presets:
- comments
- common-false-positives
- legacy
- std-error-handling
paths:
- third_party$
- builtin$
- examples$
- internal/app/fetcher/msg_hash/store.go
formatters:
enable:
- gofmt
- goimports
exclusions:
generated: lax
paths:
- third_party$
- builtin$
- examples$
11 changes: 6 additions & 5 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,11 @@ ENV TZ=Etc/UTC

RUN apt-get update && \
apt-get install -yqq \
tzdata build-essential cmake clang openssl \
libssl-dev zlib1g-dev gperf wget git curl \
libreadline-dev ccache libmicrohttpd-dev ninja-build pkg-config \
libsecp256k1-dev libsodium-dev liblz4-dev
tzdata build-essential cmake clang openssl \
autoconf libtool \
libssl-dev zlib1g-dev gperf wget git curl \
libreadline-dev ccache libmicrohttpd-dev ninja-build pkg-config \
libsecp256k1-dev libsodium-dev liblz4-dev

ADD --keep-git-dir=true https://github.com/ton-blockchain/ton.git /ton
RUN cd /ton && git submodule update --init --recursive
Expand All @@ -21,7 +22,7 @@ RUN mkdir /output && cp build/emulator/libemulator.so /output


# build
FROM golang:1.21.4-bookworm AS builder
FROM golang:1.23-bookworm AS builder

RUN apt-get update && \
apt-get install -y libsecp256k1-1 libsodium23
Expand Down
7 changes: 7 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,13 @@ Run tests on abi package:
go test -p 1 $(go list ./... | grep /abi) -covermode=count
```

To run the tests you might need to provide a path to the emulator library. For example:

```shell
CGO_LDFLAGS="-L /Users/user/go/src/github.com/tonkeeper/tongo/lib/darwin/ -Wl,-rpath,/Users/user/go/src/github.com/tonkeeper/tongo/lib/darwin/ -l emulator" \
go test -p 1 $(go list ./... | grep /abi) -covermode=count
```

Run repositories tests:

```shell
Expand Down
7 changes: 7 additions & 0 deletions abi/abi.go
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,8 @@ type InterfaceDesc struct {
ContractData TLBFieldsDesc `json:"contract_data,omitempty"`
}

var registeredDefinitions = map[TLBType]TLBFieldsDesc{}

func RegisterDefinitions(definitions map[TLBType]TLBFieldsDesc, depth ...int) error {
noDef := map[TLBType]TLBFieldsDesc{}
for dn, d := range definitions {
Expand Down Expand Up @@ -67,3 +69,8 @@ func RegisterDefinitions(definitions map[TLBType]TLBFieldsDesc, depth ...int) er

return RegisterDefinitions(noDef, currentDepth+1, maxDepth)
}

func GetRegisteredDefinition(t TLBType) (TLBFieldsDesc, bool) {
desc, ok := registeredDefinitions[t]
return desc, ok
}
Loading