Add support for the new encryption protocol #117
Conversation
This adds support for the new TP-Link discovery and encryption protocols. It is currently incomplete - only devices without username and password are current supported, and single device discovery is not implemented. Discovery should find both old and new devices. When accessing a device by IP the --klap option can be specified on the command line to active the new connection protocol.
|
|
||
| def datagram_received(self, data, addr) -> None: | ||
| """Handle discovery responses.""" | ||
| ip, port = addr | ||
| if ip in self.discovered_devices: | ||
| return | ||
|
|
||
| info = json.loads(self.protocol.decrypt(data)) | ||
| if port == 9999: |
There was a problem hiding this comment.
I would suggest that instead of relying on a hardcoded ports, to either have a command line option, or even better - attempt the old encryption mechanism, and if that fails - to try the new one.
In respect to this I've already opened a PR #109 that would extract the port and make it configurable and I think it'd quite handy for this implementation.
There was a problem hiding this comment.
Considering that there are only two discovery methods, I think it's fine to hard-code (or rather use a const from the class) for this use case.
I'd still separate the klap discovery handling into its own method that gets called only when necessary (so instead of else checking for the port for this, too), and log an error when something arrives from an unknown port.
Open question is how to handle the type detection, as the type information I saw for the new discovery responses does not give out enough information to decide between a smartplug and a strip, for example.
I'm personally for making a sysinfo query, which adds another round-trip but would allow direct reuse of the existing detection logic.
There was a problem hiding this comment.
An example discovery payload is:
{'result':
{'ip': '<ip-address>',
'mac': '<mac-address>',
'device_id': '<16 octet hex>',
'owner': '<16 octet hex>',
'device_type': 'IOT.SMARTPLUGSWITCH',
'device_model': 'HS110(UK)',
'hw_ver': '4.1',
'factory_default': True,
'mgt_encrypt_schm':
{'is_support_https': False,
'encrypt_type': 'KLAP',
'http_port': 80}
},
'error_code': 0
}
So we could use the device_model as a way of determining the different between a plug (HS100 or HS110) and a strip (KP303)
|
Thanks for the PR! Looks like a good start! Do you think you can provide some tests and fixtures too? There's some linting changes needed, but this can be handled as a last step after the code is ready to be merged. |
|
Those failures look like problems with adding the new dependencies to poetry. Is there something that needs to happen to tell the builder that they are safe? |
|
It looks like you either forgot to run If you forgot to run And commit |
|
I did poetry add in order to get the new pyproject.toml, but didn't commit poetry.lock, as it seemed like there were a load of changes to other modules in there. I'll push the new poetry.lock and see what happens! |
|
Now it fails because the changed files are not properly formatted using black. To get this to pass check lines 23 to 68 in https://github.com/python-kasa/python-kasa/blob/master/azure-pipelines.yml#L23 and execute each command. They should reformat the files so the styling "tests" can pass. |
Restore the incorrectly commented out old discovery mechanism, so both old and new devices are found.
c1136a3 to
901b0ce
Compare
Codecov Report
@@ Coverage Diff @@
## master #117 +/- ##
==========================================
- Coverage 73.02% 68.00% -5.02%
==========================================
Files 10 12 +2
Lines 1227 1394 +167
Branches 183 203 +20
==========================================
+ Hits 896 948 +52
- Misses 298 412 +114
- Partials 33 34 +1
Continue to review full report at Codecov.
|
|
Thanks @kirichkov I think I finally got there in terms of the tests. |
|
@rytilahti Can you take a look as well? Should we merge #109 before that to refactor this on top of it or do it the other way around? |
Add the --user and --password command line options which can be used to provide a username and password to authenticate to the smart device using the KLAP protocol. Check the 'owner' field in the discovery packet, and only attempt to authenticate to devices for which we have a password.
|
mypy running locally is happy with this, so I'm not sure what the problem is. |
kasa/protocol.py
Outdated
| try: | ||
| session = aiohttp.ClientSession(cookie_jar=self.jar) | ||
|
|
||
| async with self.handshake_lock: |
There was a problem hiding this comment.
I think this lock should be generic for all requests, to avoid potential reuse of the seq number in case several queries are done concurrently?
kasa/protocol.py
Outdated
| signature = hashlib.sha256( | ||
| self.hmacKey + seq.to_bytes(4, "big", signed=True) + ciphertext | ||
| ).digest() | ||
| return signature + ciphertext |
There was a problem hiding this comment.
| return signature + ciphertext | |
| return signature + ciphertext |
Please separate exit points with an empty line for readability.
kasa/protocol.py
Outdated
| if resp.status != 200: | ||
| raise SmartDeviceException( | ||
| "Device responded with %d to handshake1" % resp.status | ||
| ) |
There was a problem hiding this comment.
| ) | |
| ) | |
I'd separate the exit point here with an empty line.
Add a new class which holds authentication information for the KLAP protocol
| def __init__(self, host: str): | ||
| super().__init__(host=host) |
There was a problem hiding this comment.
| def __init__(self, host: str): | |
| super().__init__(host=host) |
| raise SmartDeviceException("Not reached") | ||
|
|
||
| async def _ask(self, request: str) -> str: | ||
| raise SmartDeviceException("ask should be overridden") |
There was a problem hiding this comment.
raise NotImplementedError like SmartDevice's "abstract" methods do.
| @@ -255,7 +263,7 @@ async def _query_helper( | |||
| request = self._create_request(target, cmd, arg, child_ids) | |||
|
|
|||
| try: | |||
| response = await self.protocol.query(host=self.host, request=request) | |||
| response = await self.protocol.query(request=request) | |||
There was a problem hiding this comment.
| response = await self.protocol.query(request=request) | |
| response = await self.protocol.query(request) |
I think in this case it is not necessary to use kwarg, it is pretty clear already what is going in :-)
| @@ -300,7 +308,7 @@ async def update(self): | |||
| # Check for emeter if we were never updated, or if the device has emeter | |||
| if self._last_update is None or self.has_emeter: | |||
| req.update(self._create_emeter_request()) | |||
| self._last_update = await self.protocol.query(self.host, req) | |||
| self._last_update = await self.protocol.query(request=req) | |||
There was a problem hiding this comment.
| self._last_update = await self.protocol.query(request=req) | |
| self._last_update = await self.protocol.query(req) |
| async def _handshake(self, session) -> None: | ||
| _LOGGER.debug("[KLAP] Starting handshake with %s", self.host) | ||
|
|
||
| # Handshake 1 has a payload of client_challenge |
There was a problem hiding this comment.
Please separate handling of different handshake phases to their own methods, that will make it easier to write unit tests.
| owner_bin, | ||
| self.emptyUser, | ||
| ) | ||
| if owner is None or owner == "" or owner_bin == self.emptyUser: |
There was a problem hiding this comment.
The owner cannot be none anymore at this point, so checking if the owner == Auth.empty_user should suffice?
| _LOGGER.debug("[DISCOVERY] Device %s has no owner", ip) | ||
| device = device_class(ip, Auth()) | ||
| elif ( | ||
| self.authentication is not None |
There was a problem hiding this comment.
The self.authentication needs to be already there if we get this far, so checking for the owner should be enough.
| https://github.com/softScheck/tplink-smartplug/ | ||
|
|
||
| which are licensed under the Apache License, Version 2.0 | ||
| http://www.apache.org/licenses/LICENSE-2.0 |
There was a problem hiding this comment.
This module docstring was valid only for the legacy implementation, please adapt (or remove altogether?).
| agreed = self.client_challenge + self.server_challenge + self.authenticator | ||
| self.encrypt_key = self._sha256(b"lsk" + agreed)[:16] | ||
| self.hmac_key = self._sha256(b"ldk" + agreed)[:28] | ||
| fulliv = self._sha256(b"iv" + agreed) |
There was a problem hiding this comment.
Group local & instance assignments separately, consider adding some newlines to make it nicer to read.
| # In theory we should verify the hmac here too | ||
| return Padding.unpad(cipher.decrypt(payload[32:]), AES.block_size) | ||
|
|
||
| async def _ask(self, request: str) -> str: |
|
Just wanted to check if this is likely to move forward and be integrated? |
I need to find the time to complete the necessary cleanup and testing. It's at a point where it works fine for day to day use for me, and so it has dropped down the priority list somewhat. My main goal in doing this work was Home Assistant integration (I'm using roughly the same changes against pyHS110), so this is less urgent for me until the Home Assistant TP-Link library moves over to python-kasa. |
My apologies I did not want to add any pressure on you. I will play with your fork as a starting point and take it from there, it is my goal to get this working in homebridge. I think that will require a new plugin based on python-kasa as that also does not currently use it (as far as I am aware). May I ask or may I email you (or you can email me [email protected]) regarding your basic setup using your version? |
Making an emeter request against an HS100 plug crashes it. So don't.
|
What’s needed to complete this item? |
As @SimonWilkinson stated in #117 (comment), its not a priority until Home Assistant moves over to using this library, which isn't likely to happen until October. |
|
Heya Just wanted to say thank you to the work put in by @SimonWilkinson and @ghostseven for their updates to this! I have been playing around behind the scenes, when I can, using both your forks to test out my HS110(UK) v4.1 (v1.1.0 FW) plugs and ran into all sorts of issues, eventually realised it was down to my own fault of using the wrong password in the command like to run the discovery. (Although resetting one of my plugs to "Local Mode" meant I was able to access without the username/password). @bdraco Looks like you were able to get the integration working with this module (awesome work too!), and it looks like everything is falling into place! I am going to attempt to see if I can work my HA around to use this fork of the python module to see if I can get anywhere with assisting with this, but before I wonder down a rabbit hole (or step on anyone's toes/add additional work for anyone here) - is there anything anyone would like me to have a look at / or stay away from? I don't want to add any more pressure onto anyone here, but I would like to try and do whatever I can to help with this! Thanks again for all your help and support getting through these annoying changes! |
|
I don't have any of the affected devices so I'm not able to test this. |
|
Just to keep people up to date here, @Yamakiroshi and I have been working on and off trying to just "hack" this back into a working state, both with the Kasa CLI and through Home Assistant, and as of a few moments ago we succeeded! There has been a lot of really bad coding to make this all work, but I will try and share as we push through this. One key element is we took @SimonWilkinson latest forked, forked that again, merged in all the current code from the main branch here, along with some very "non-best practice" commits just to try and get the whole thing working. The unsorted code for this repo is currently here - This needs a lot of work, but using changes here, and some hacks to the It is very late here, so I am going to have to drop off for the night, but I will have a go at cleaning up the code as much as possible, try and bring in some of the code review comments from here, and then see what needs to be done next. I am not an expert at this at all, so would really appreciate any support to see if we can get this over the line. Thank you again to everyone for the work so far! |
|
It will be a lot easier to give feedback once a PR is opened, even if its in a very raw form.
… On Nov 16, 2021, at 18:34, Colin Barker ***@***.***> wrote:
Just to keep people up to date here, @Yamakiroshi <https://github.com/Yamakiroshi> and I have been working on and off trying to just "hack" this back into a working state, both with the Kasa CLI and through Home Assistant, and as of a few moments ago we succeeded! There has been a lot of really bad coding to make this all work, but I will try and share as we push through this.
One key element is we took @SimonWilkinson <https://github.com/SimonWilkinson> latest forked, forked that again, merged in all the current code from the main branch here, along with some very "non-best practice" commits just to try and get the whole thing working.
The unsorted code for this repo is currently here <https://github.com/TokonatsuFestival/python-kasa/tree/ha-fixes> - This needs a lot of work, but using changes here, and some hacks to the tplink custom component, we got control and power usage, using the new KLAP protocol.
It is very late here, so I am going to have to drop off for the night, but I will have a go at cleaning up the code as much as possible, try and bring in some of the code review comments from here, and then see what needs to be done next. I am not an expert at this at all, so would really appreciate any support to see if we can get this over the line.
Thank you again to everyone for the work so far!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub <#117 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFB7CF372FKMAQSXWWYRQDUML2BNANCNFSM4TYVKZ2Q>.
|
@mystcb can you do a PR? |
|
Apologies all, had a few things come up - let me get the PR in - I am going to reset everything and pull all the changes into one place so it is all up to date. I will get this done ASAP PR Raised: #267 |
|
#509 is now merged so I'm closing this as obsolete, thanks @SimonWilkinson! |
Here's a very rough implementation of the new encryption protocol.
Tested against an HS110 running the new firmware.
I haven't written much python recently, and even less using the asyncio framework, so comments and criticism very welcome.
Things that I know work:
*) Discovery
*) Most appropriate command line options when paired with the '--plug' option
Username and password support is still required, but should be simple. Single device discovery support is also not yet complete.
Fixes #170 fixes #115