Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Collect network statistics and make it available for the user applications. #8310

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Nov 12, 2024

Conversation

hvlad
Copy link
Member

@hvlad hvlad commented Nov 7, 2024

The feature makes Remote provider to collect various network statistics counters and let user applications to query it using IAttachment::getInfo() (isc_database_info()) API.

There are nine counters defined:

	fb_info_wire_out_packets = 150,
	fb_info_wire_in_packets = 151,
	fb_info_wire_out_bytes = 152,
	fb_info_wire_in_bytes = 153,
	fb_info_wire_snd_packets = 154,
	fb_info_wire_rcv_packets = 155,
	fb_info_wire_snd_bytes = 156,
	fb_info_wire_rcv_bytes = 157,
	fb_info_wire_roundtrips = 158,

Counters with "in" and "out" in its names forms "logical IO" group and describes number of packets and bytes passed from the Firebird protocol point of view.

Counters with "snd" and "rcv" in its names forms "physical IO" group and describes number of packets and bytes passed over the physical connection. Also, there is fb_info_wire_roundtrips counter in "physical IO" group that shows how many times IO direction was changed from 'SEND' to 'RECEIVE'.

If wire compression is active, values of physical bytes counters should be less than values of corresponding logical counters.
If wire compression is not active, both physical bytes counters and logical bytes counters should be equal.

While counters accumulated on both sides of the connection, only client side is available for client application:

  • it should be equal on both sides, except of opposite IO direction and fb_info_wire_roundtrips value,

  • it requires no network rountdrip to query client side (i.e. local) counters.

    For embedded connection: wire statistics is not exists and engine implementation of IAttachment::getInfo() doesn't handle wire stats items. It returns isc_info_error, isc_infunk in the response for every wire stats item. It could be changed to return zero's, if necessary.

    isql utility got new commands to show wire statistics. It is described in doc\README.isql_enhancements.txt

@hvlad hvlad self-assigned this Nov 7, 2024
@aafemt
Copy link
Contributor

aafemt commented Nov 7, 2024

  • For simplicity, it is not allowed to mix locally known info items (i.e. enlisted above) with other info items in the single call of IAttachment::getInfo().

Why? MERGE_database_info() already handle a bunch of items. Adding some more shouldn't be hard.

@aafemt
Copy link
Contributor

aafemt commented Nov 7, 2024

This is an improvement. Shouldn't it to be applied to master branch only/first?

@hvlad
Copy link
Member Author

hvlad commented Nov 7, 2024

  • For simplicity, it is not allowed to mix locally known info items (i.e. enlisted above) with other info items in the single call of IAttachment::getInfo().

Why? MERGE_database_info() already handle a bunch of items. Adding some more shouldn't be hard.

To not pass wire stats items into the next layer that anyway have no idea about it.

@hvlad
Copy link
Member Author

hvlad commented Nov 7, 2024

This is an improvement. Shouldn't it to be applied to master branch only/first?

Usually - yes.
With this case it is completely new feature and should not affect any existing functionality.
And I see no reason why it should not be in v5 also.

But, formally, you are correct, I agree.

@dyemanov
Copy link
Member

dyemanov commented Nov 7, 2024

I'd expect these stats (its server-side counterpart) to be also available in some new MON$ table. Not in v5 for sure.

@hvlad
Copy link
Member Author

hvlad commented Nov 7, 2024

I'd expect these stats (its server-side counterpart) to be also available in some new MON$ table. Not in v5 for sure.

Agree

@AlexPeshkoff
Copy link
Member

  • For simplicity, it is not allowed to mix locally known info items (i.e. enlisted above) with other info items in the single call of IAttachment::getInfo().

Why? MERGE_database_info() already handle a bunch of items. Adding some more shouldn't be hard.

To not pass wire stats items into the next layer that anyway have no idea about it.

This appears to be suspicious reason. It's pretty easy to remove remote provider specific items before sending info request to server. And as it was aslready mentioned merging two sets of info items is also simple task. So where is simplicity with suggested approach?

@hvlad
Copy link
Member Author

hvlad commented Nov 7, 2024

  • For simplicity, it is not allowed to mix locally known info items (i.e. enlisted above) with other info items in the single call of IAttachment::getInfo().

Why? MERGE_database_info() already handle a bunch of items. Adding some more shouldn't be hard.

To not pass wire stats items into the next layer that anyway have no idea about it.

This appears to be suspicious reason. It's pretty easy to remove remote provider specific items before sending info request to server. And as it was aslready mentioned merging two sets of info items is also simple task. So where is simplicity with suggested approach?

MERGE_database_info() is called after sending of op_info_database to the remote server. Thus is makes no help if we want and able to avoid network exchange.

As already said, this requirement was made to keep code simple. If you insist, I can add more complexity here.
And - for nothing - as it is far not expected to mix wire stats items with something else in same request.
It can't affect existing code as there was no wire stats items before and there is no reason to add them into existing requests.

@aafemt
Copy link
Contributor

aafemt commented Nov 7, 2024

One loop through the request. For every local item fill the answer buffer and remove the item from the request. Then if request is not empty and answer buffer has some space - send request and append received answer to the answer buffer. I see no complexity here.

@hvlad
Copy link
Member Author

hvlad commented Nov 7, 2024

One loop ...

Do you really think I don't know how to do it ?
BTW, what role plays MERGE_database_info() in your new suggestion ?

I see no complexity here.

Good for you.

@hvlad
Copy link
Member Author

hvlad commented Nov 8, 2024

Requirement to not mix local and remote handled items in the same info request is removed.
Description changed accordingly.

@hvlad hvlad merged commit 247cacc into v5.0-release Nov 12, 2024
44 of 48 checks passed
hvlad added a commit that referenced this pull request Nov 12, 2024
…tions. (#8310)

* Make Remote provider collect wire statistics.
New info items to query wire stats counters.

* New ISQL commands to show wire statistics.

* Remove requirement to not mix local and remote handled items in the same info request.

* Follow @asfernandes suggestion about class members initialization.

* Make ctor explicit, as @sim1984 suggested

* Put isc_info_end into response buffer despite of its presence in info items.
@sim1984
Copy link

sim1984 commented Nov 13, 2024

HELP SET command does not display help for SET WIRE_STAT

SQL> help set;
Set commands:
    SET                    -- display current SET options
    SET AUTOddl            -- toggle autocommit of DDL statements
    SET BAIL               -- toggle bailing out on errors in non-interactive mode
    SET BLOB [ALL|<n>]     -- display BLOBS of subtype <n> or ALL
    SET BLOB               -- turn off BLOB display
    SET COUNT              -- toggle count of selected rows on/off
    SET MAXROWS [<n>]      -- limit select stmt to <n> rows, zero is no limit
    SET ECHO               -- toggle command echo on/off
    SET EXPLAIN            -- toggle display of query access plan in the explained form
    SET HEADING            -- toggle display of query column titles
    SET KEEP_TRAN_params   -- toggle to keep or not to keep text of following successful SET TRANSACTION statement
    SET LIST               -- toggle column or table display format
    SET NAMES <csname>     -- set name of runtime character set
    SET PER_TABle_stats    -- toggle display of detailed per-table statistics
    SET PLAN               -- toggle display of query access plan
    SET PLANONLY           -- toggle display of query plan without executing
    SET SQL DIALECT <n>    -- set sql dialect to <n>
    SET STATs              -- toggle display of performance statistics
    SET TIME               -- toggle display of timestamp with DATE values
    SET TERM <string>      -- change statement terminator string
    SET WIDTH <col> [<n>]  -- set/unset print width to <n> for column <col>

All commands may be abbreviated to letters in CAPitals

@hvlad
Copy link
Member Author

hvlad commented Nov 13, 2024

HELP SET command does not display help for SET WIRE_STAT

WIll add it, thanks

@hvlad hvlad deleted the work/wire_stats branch November 14, 2024 18:57
@hvlad
Copy link
Member Author

hvlad commented Nov 14, 2024

HELP SET command does not display help for SET WIRE_STAT

WIll add it, thanks

Done

@pavel-zotov
Copy link

::: QA note :::
Test currently verifies only ability to obtain in ISQL wire counters and statistics as described in the doc.
More complex test(s) will be implemented after firebird-driver become to recognize appropriate API changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants