-
Notifications
You must be signed in to change notification settings - Fork 2.2k
FINERACT-2314: IP tracking #4825
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
@adamsaghy content has been added. |
|
@JohnAlva Please rebase your PR and make sure you run the spotlessApply and checkstyle on the PR before it got committed: |
|
|
||
| private final FineractProperties fineractProperties; | ||
|
|
||
| private static final String[] IP_HEADER_CANDIDATES = { "X-Forwarded-For", "Proxy-Client-IP", "WL-Proxy-Client-IP", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How was the priority order of the IP header candidates decided?
Is there a specific reason for the ordering, or is it based on best practice or precedent?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@JohnAlva Can you please advise on this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We have used the IP headers precedence based on these documents:
https://www.rfc-editor.org/rfc/rfc7239.txt
https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/X-Forwarded-For
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@JohnAlva These articules only talks about "X-Forwarded-For"... what about the rest of them???
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
big no. This must be explained one by one why a specific header is used. Please put the comments in the code so everybody understands the purpose.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done, comments added as requested.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for the detailed comments, but please make sure you are split them into separate lines and please drop the //.
Example:
X-Forwarded-For: Standard header used by proxies,
Proxy-Client-IP: Used by some Apache proxies,
...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Apologies for the delayed response, the change has already been made.
| for (String header : IP_HEADER_CANDIDATES) { | ||
| String ip = request.getHeader(header); | ||
| if (ip != null && ip.length() != 0 && !"unknown".equalsIgnoreCase(ip)) { | ||
| log.debug("SEND IP : {}", ip); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"SEND IP" you mean "CALLER IP"?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, I've changed it to 'CALLER IP'.
| try { | ||
| String clientIpAddress = getClientIpAddress(request); | ||
| if (StringUtils.isNotBlank(clientIpAddress)) { | ||
| log.info("Found Client IP in header : {}", clientIpAddress); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please change the log level to DEBUG
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, I've changed the log level to DEBUG.
...e/src/main/java/org/apache/fineract/infrastructure/core/filters/GeolocationHeaderFilter.java
Outdated
Show resolved
Hide resolved
| } | ||
| filterChain.doFilter(request, response); | ||
| } catch (Exception e) { | ||
| e.printStackTrace(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please have correct error handling! "e.printStackTrace" is incorrect and easily hide exceptions that propagate from other filters!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We've implemented the correct error handling as requested. Instead of e.printStackTrace(), we've ensured that the exception is properly logged, and any exceptions are now handled in a way that doesn't hide issues from other filters.
| final String clientName = rs.getString("clientName"); | ||
| final String loanAccountNo = rs.getString("loanAccountNo"); | ||
| final String savingsAccountNo = rs.getString("savingsAccountNo"); | ||
| final String ip = (rs.getString("ip") != null) ? rs.getString("ip") : "SN/IP"; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what does "SN/IP" means?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've changed it to 'NO IP' for better understanding
...e/src/main/java/org/apache/fineract/infrastructure/core/filters/GeolocationHeaderFilter.java
Outdated
Show resolved
Hide resolved
| .addFilterAfter(fineractInstanceModeApiFilter(), CorrelationHeaderFilter.class); // | ||
| .addFilterAfter(fineractInstanceModeApiFilter(), CorrelationHeaderFilter.class) // | ||
| .addFilterAfter(fineractInstanceModeApiFilter(), CorrelationHeaderFilter.class) // | ||
| .addFilterAfter(geolocationHeaderFilter(), RequestResponseFilter.class); // |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You can decide to not register this filter if the functionality is disabled... See example: loanCOBFilterHelper
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We've added a function to handle the filters when they are disabled. Specifically, we implemented this for the geolocation filter, as it was not previously disabled, and we have now included this functionality.
| fineract.correlation.header-name=${FINERACT_LOGGING_HTTP_CORRELATION_ID_HEADER_NAME:X-Correlation-ID} | ||
|
|
||
| fineract.job.stuck-retry-threshold=${FINERACT_JOB_STUCK_RETRY_THRESHOLD:5} | ||
| fineract.geolocation.enabled=${FINERACT_GEOLOCATION_ENABLED:true} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I dont think we should enabled this by default... but i can be convinced...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We've set it to false instead, as it seems more appropriate for this case.
|
@adamsaghy Hi Adam |
|
@JohnAlva the PR in conflicted state. the tests are failed and there are great many open questions on it! |
|
@adamsaghy Hello, Can you help us review again, and about the change of tests we don't work on those. |
|
@JohnAlva Please use the correct PR title and commit message naming convention: |
adamsaghy
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please check my comment regarding the PR title and commit message!
Please check all the unanswered comments and advise on them!
Please make sure, you are adding proper testing which ensures the functionality is working as expected!
f1f890c to
d4ae4fc
Compare
|
@galovics Hey, could you help us review the commit and let us know if you have any comments? |
| this.loanExternalId = result.getLoanExternalId(); | ||
| } | ||
|
|
||
| private static String getClientIp() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please move this logic outside of the JPA entity.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I’ve moved the getClientIp() logic out of the JPA entity and placed it inside the JsonCommand class. Let me know if any other changes are needed
| private IdempotencyStoreHelper idempotencyStoreHelper; | ||
|
|
||
| @Autowired(required = false) | ||
| private GeolocationHeaderFilter geolocationHeaderFilter; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would CallerIpTrackingFilter naming make more sense here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good point — renamed to CallerIpTrackingFilter for clarity.
adamsaghy
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Kindly review my comments!
Please make sure you are adding proper testing that ensures this new logic works as expected!
|
@adamsaghy Thanks for the reminder! I’ll make sure to add proper tests to verify the new logic behaves as expected. |
|
@adamsaghy Just checking—do you need any input on the test scenarios or coverage, or are you all set? |
I am not sure what you mean... 🤔 |
|
Hi @adamsaghy, |
4520ced to
1a7b85c
Compare
I will be on leave for the rest of the week, but example for integration test: AuditIntegrationTest (i believe this is fetching the content of the Also for E2E test, we might not yet created any AUDIT related one, but you can take a look at any of them and see how it works... |
a96b30c to
0a3cf82
Compare
| if (!ip.isEmpty()) { | ||
| assertEquals(EXPECTED_LOCAL_IP, ip); | ||
| } else { | ||
| assertEquals("", ip); | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How can there be an IF in the assertion? This seems to be wrong.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have already made the correction by removing the if inside the assertion, as you suggested. Now the assertion is directly performed after verifying that the IP is not empty.
| fineract.correlation.header-name=${FINERACT_LOGGING_HTTP_CORRELATION_ID_HEADER_NAME:X-Correlation-ID} | ||
|
|
||
| fineract.job.stuck-retry-threshold=${FINERACT_JOB_STUCK_RETRY_THRESHOLD:5} | ||
| fineract.client-ip-tracking.enabled"=${FINERACT_CLIENT-IP-TRACKING_ENABLED:false} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Use underscores in the env var name.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done.
| @ConditionalOnProperty("fineract.client-ip-tracking.enabled") | ||
| @RequiredArgsConstructor | ||
| @Slf4j | ||
| public class CallerIpTrackingFilter extends OncePerRequestFilter { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't get the whole filter approach.
If this is truly just extracting the IP info from the servlet request, why don't you simply create a utility method for it in a utility class and use it when needed? Why do you change the attributes of the request?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It was implemented as a filter because it is considered an optional value. This allows the system user to decide whether or not they want to capture the IP address.
If it were implemented as a Utils class, the IP capture would be executed automatically at all times, without the possibility of skipping it.
Additionally, we believe it is not necessary to create an additional class solely for this purpose, as it would require more development effort for a functionality that can remain optional more easily through a filter.
|
|
||
| private FineractCorrelationProperties correlation; | ||
|
|
||
| private FineractGeolocationProperties geolocation; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You changed the property name, it's not geolocation anymore.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done, I have changed the name to ipTracking.
| public String getClientIp() { | ||
| ServletRequestAttributes attrs = (ServletRequestAttributes) RequestContextHolder.getRequestAttributes(); | ||
| String clientIp = ""; | ||
| if (attrs != null) { | ||
| Object ipAttr = attrs.getRequest().getAttribute("IP"); | ||
| if (ipAttr != null) { | ||
| clientIp = ipAttr.toString(); | ||
| } | ||
| } | ||
| return clientIp; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I meant the utility method here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As mentioned, the filter keeps IP tracking optional. Using a utility class would make capture automatic, with no option to skip. Creating a separate class for this would add unnecessary overhead, as the filter already fulfills the required functionality.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Shelaslifter Fetching data from RequestContextHolder.getRequestAttributes() has nothing to do with JsonCommand class. Please Move this while logic into a utility class as @galovics recommended.
We should not mix business logics together...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@adamsaghy hi, the IpAddressUtils.java class was created, and the JsonCommand was removed from the class.
de543f3 to
4b641ac
Compare
|
@galovics Hi, could you please help us review the commit and share any feedback or comments you may have? Thanks. |
|
@galovics Hey, would you mind reviewing the commit and providing any feedback or thoughts you might have? Appreciate it. |
| import org.junit.jupiter.api.BeforeEach; | ||
| import org.junit.jupiter.api.Test; | ||
|
|
||
| public class IpTrackingIntegrationTest { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would you mind to write a negative test case, where functionality is not enabled, so there will be no IP fetched and stored?
Also Please make sure, all of the IP header options are tested one by one... ;)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The one for the headers was already added, and the functionality was moved to IpAddressUtils.java.
540fcf2 to
650a48d
Compare
1fa9de8 to
916481e
Compare
f435304 to
4cbc1c9
Compare
adamsaghy
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Co-authored-by: Juan Pablo Alvarez Hernandez <work_jpa@hotmailcom>
This quality of life patch reduces static weaving log message priority,
reducing the default gradle build output by about 60 lines.
example gradle run with these messages:
> Configure project :custom
ℹ Skipping static weaving configuration for non-Java project: custom
> Configure project :fineract-accounting
Configuring EclipseLink static weaving for fineract-accounting
> Configure project :fineract-branch
Configuring EclipseLink static weaving for fineract-branch
...
* update release notice years in NOTICE_RELEASE and NOTICE_SOURCE
* step 5
* simplify mention of tests: "Ensure all tests pass for this commit both in CI and locally"
* recommend GPG signing annotated release tag
* step 8: improve svn commands
* This way is simpler and more efficient, especially with a bunch of release candidate dirs in the staging area.
* step 10: mention need to test rc before +1 vote -- See:
* https://www.apache.org/legal/release-policy.html#release-approval
* https://www.apache.org/legal/release-policy.html#approving-a-release
* step 12: only PMC members can upload releases
* step 13: simplify & explain "finalize branch" instructions
* Document what worked for me for 1.12.1.
* I didn't need to create the extra `merge-$VERSION` branch and do the recursive merge.
* document how asciidoctor upgrade is blocked
* see "official docs - deps stuck" thread on fineract dev mailing list
* https://lists.apache.org/thread/7mmsj13spb11vgz0z38fhwgzwtq03brr
* can't upgrade to 4.x because of one of these:
* asciidoctor/asciidoctorj-pdf#25
* jruby/jruby#5573
* asciidoctor/asciidoctorj-pdf#16
* improve asciidoc config - opt for simplicity where the complexity adds nothing
* compat-mode is off by default, no need for it here
* default optimization should be fine
* media should have been screen | print | prepress, just leave it as default instead
* page size? I really don't think this is going to be printed much, just go with default
* PDF version 1.8 is invalid, just use the default unless we someday have a good reason to pin this
* reduce copyright years sources of truth
* see also: https://docs.asciidoctor.org/pdf-converter/latest/asciidoc-attributes/
* remove unnecessary asciidoctorj 3.0.0 version string -- no need to pin this
* remove prompt character from Bash shell examples
* it isn't properly syntax-highlighted and it looks confusing with line numbers (which we might want to add)
* it isn't necessary
* the prompt character ("%" in this case) is not typically included in shell code examples because it makes it harder to copy and paste shell code examples
* $ is likely more common than % (at least on Debian/Ubuntu), but either way I'd say exclude it
* fix source code syntax labels - use "bash" only when it is actually Bash shell code
* persistence.adoc
* fix broken enumerated list
* resolves these warnings seen with, e.g.: `gradle --info doc`
* `Jul 27, 2025 8:26:48 PM uri:classloader:/gems/asciidoctor-2.0.10/lib/asciidoctor/parser.rb parse_list_item`
* `WARNING: chapters/architecture/persistence.adoc: line 104: list item index: expected 1, got 2`
* `Jul 27, 2025 8:26:48 PM uri:classloader:/gems/asciidoctor-2.0.10/lib/asciidoctor/parser.rb parse_list`
* `WARNING: chapters/architecture/persistence.adoc: line 110: list item index: expected 1, got 3`
* fix wrapping (we use hardbreaks)
* fix typo: `s/plane text/plain text/`
* switch to rouge syntax highlighter - it handles more source languages
* fix broken long shell code lines
* fix .avro file syntax highlighting (it's JSON)
* configuration-gpg.adoc
* fix accidental block continuation
* One little plus sign was making `= Email` appear verbatim in rendered output because it was interpreted as a list continuation.
* See https://docs.asciidoctor.org/asciidoc/latest/lists/continuation/#list-continuation
* recommend more secure keys
* add a line continuation for an enumerated list
* architecture-overview.puml: remove this unused (likely a "Hello World") diagram
* release-schedule.puml: fix pluralization of days
* purely aesthetic: doesn't affect chart rendering
* fix src/bin/binary tarball name typos
* missed a few in e090da2
* fix release branch name
* must match `release/{revnumber}`, per gitVersioning stanza in top level build.gradle
* harden.adoc: fix broken link to CISA
* fineract-doc/build.gradle
* ensure HTML task has diagrams and images availble
FINERACT-2317: Add documentation for approved amount modification endpoints
Bump actions/cache from 4.2.3 to 4.2.4
Bumps [actions/cache](https://github.com/actions/cache) from 4.2.3 to 4.2.4.
- [Release notes](https://github.com/actions/cache/releases)
- [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md)
- [Commits](actions/cache@5a3ec84...0400d5f)
---
updated-dependencies:
- dependency-name: actions/cache
dependency-version: 4.2.4
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <[email protected]>
FINERACT-2314: IP tracking (apache#4825)
Co-authored-by: Juan Pablo Alvarez Hernandez <work_jpa@hotmailcom>
FINERACT-2340: remove nonexistant project "fineract-api" (apache#4924)
FINERACT-2326: The journal entries should be ordered in explicit order
FINERACT-2338: Allow backdated interest change on progressive loans - documentation
FINERACT-2326: Improve null-safety
FINERACT-2338: Allow backdated interest change on progressive loans
- charge-off handling
- write-off handling
- closed loans
- external owner changes
FINERACT-2326: Tax component and group issue serialization
FINERACT-2326: Charges with Tax group Id ignored
FINERACT-2323: support the multiple legs for journal entries
FINERACT-2326: [DOC] Interest rate change documentation
FINERACT-2324: Remove getLoanTransactions from accounting
FINERACT-2326: Fix swagger generation
FINERACT-2343: Fix update currency api validation
FINERACT-2343: added e2e tests for business date and currency validation
FINERACT-2326: Improve command query param regexp to allow hyphens
FINERACT-2326: Fix swagger generation
FINERACT-2326: Fix swagger generation
Bump actions/checkout from 4.2.2 to 5.0.0
Bumps [actions/checkout](https://github.com/actions/checkout) from 4.2.2 to 5.0.0.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](actions/checkout@11bd719...08c6903)
---
updated-dependencies:
- dependency-name: actions/checkout
dependency-version: 5.0.0
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <[email protected]>
FINERACT-2326: Extract out external dependencies from Client and Group entity
FINERACT-2330: Buy-Down fees Accounting for non merchant product
FINERACT-2326: Missing user permission for Capitalized Income and Buydown fee
FINERACT-2181: Update dependency node to v22
FINERACT-2181: Update dependency com.puppycrawl.tools:checkstyle to v11
FINERACT-2326: Fix UserLoanPermissionTest
FINERACT-2326: Use Hibernate Validator
FINERACT-2232: DeferredIncomeApi as CapitalizedIncomeApi
FINERACT-2326: Introduce FineractProgressiveLoanBeanConfiguration to allow conditionally register beans in `fineract-progressive-loan` module
FINERACT-2326: Rework business date validation and dto handling
FINERACT-2233: Rework journal entry handling logic in Loan module
FINERACT-2279: Add contract termination documentation
FINERACT-1981: Reschedule loan with interest rate change from zero breaks repayment schedule and loan status to OVERPAID
FINERACT-1981: Reschedule loan with interest rate change from zero breaks repayment schedule and loan status to OVERPAID - E2E tests
FINERACT-2312: Adjustment to savings account products by adding a new accounting account interest receivables account. (apache#4876)
FINERACT-2312: Accruals added for savings accounts (apache#4885)
cambios para reverse
test
Description
FINERACT-2314: IP tracking
Ignore if these details are present on the associated Apache Fineract JIRA ticket.
Checklist
Please make sure these boxes are checked before submitting your pull request - thanks!
FYI our guidelines for code reviews are at https://cwiki.apache.org/confluence/display/FINERACT/Code+Review+Guide.