Failed Execution of Standard Schedule Not Recorded in WW Logs

duplada

I run a number of standard repeating schedules that run overnight 7 days a week.

Last night a charge schedule " Part 2 Charge" failed to execute @ 03:27 and it did not leave any reason or trace in the WW log files. I have had this happen once before and the next day it ran as normal without any modification. The schedule is a straight forward charge schedule to 100% with no conditions set.

If you could take a look for me please circular- strategy

admin

duplada Looks like the system data response from Giv was incomplete and we could not parse it due to the many null values, including things like SOC/percent.

Error parsing system data response: {"data":{"time":null,"status":"Unknown","solar":{"power":null,"arrays":[{"array":1,"voltage":null,"current":null,"power":null},{"array":2,"voltage":null,"current":null,"power":null}]},"grid":{"voltage":null,"current":null,"power":null,"frequency":null},"battery":{"percent":null,"power":null,"temperature":null},"inverter":{"temperature":null,"power":null,"output_voltage":null,"output_frequency":null,"eps_power":null},"consumption":null}}, txId: BGIrdHim

We will fix the bug and ensure this shows up in the activity log. And will be added to alerts.

duplada

That's a lot of Null's - Great Support Thank you

duplada

So because my SOC was around 17% this evening my export schedules Quick Discharge 1 - 4 didn't execute which is as expected. When WW executed "Quick Discharge 5" @ 22:50 which takes the battery down to 4% it errored out with the following. Looks as if the SOC % is missing.

I will watch the system to see if my first Charge Schedule "Part 1 Charge" executes correctly @ 23:30 and report back. If you can take another look at it please.

admin

duplada Yep, same issue, null values returned from Giv API, which we cannot do anything with. We need SOC.

At least we now know the log fix is working!

duplada

Schedule "Part Charge 1" executed correctly @ 23:30

duplada

I will see if "Part 1 Discharge" schedule executes @ 01:40 and let you know

duplada

So have GE updated their API and have an issue with it? It seems rather strange it has happened twice in 24 hours to me but not to other users.

This has to be some sort of intermittent problem as the charge schedule which errored when I first reported the problem ran fine last night, but an export schedule that worked fine the day before errored last night, Neither have been amended. The three schedules that have run since the last error have all succeeded

When that type of error is received, would it not be possible to retry the execution again?

What is the way forward here, Do you have contacts at GE to explore this further as a consistently reliable system is essential for user confidence.

Keep me updated and I will let you know if I see further instances of this issue. Thanks for looking at this on a weekend.

duplada

So I had another parsing failure in the early hours today

Can you give me some insight into the operation of WW as I have been trawling through my data logs to see if I can identify any communication problems or missing data in my GE logs. My dual AIO system records data about every 60 seconds and not the usual 5 minutes. In the past occasionally I have seen odd instances where some field data has been missing from a record in the GE logs.

Is the error happening as a result of querying the GE data to see if the Schedule should be executed?
or Is the error the result of actually trying to set up the schedule on the GE server.?
what time window before a schedule time should I be looking for missing SOC data 2 of 3 minutes?

Any information that you can provide would be useful as I am determined to get things operating successfully

Would it be possible where you get a failure like the parsing error for general schedules to perform a retry a few minutes later? Retry on Failure Y/N?

admin

duplada Sure, the API call is basically:

HTTP GET:
https://api.givenergy.cloud/v1/inverter/:serial/system-data/latest

Where :serial param is your GW serial number. And you will need to sent your API token as a Bearer authentication header. You can try this with various tools like Postman.

This seems to occasionally result in a bunch of null values. But because it's a valid JSON document and we get a valid HTTP status (200 OK) back, we don't retry these. This is ultimately something that Giv will need to sort out on why their dual AIO GW sometimes responds with all null values for that specific API call (system-data/latest).

Hope that makes sense!

duplada

admin Well it sort of made sense. Both you and in particular me both know that GE is not going to sort anything out, as they just cannot cope, they won't even answer the phones these days. I have multiple open support tickets going all the way back to March. There is one person who knows and understands the dual AIO's and that is it at GE period.

I fully realise that you have a system that supports multiple inverter manufacturers and you trying to make your system as flexible and reliable as possible.

I have had 4 of these show stopping errors now. 1 was an Inverter Timeout and the other 3 have been the parsing errors where the SOC is missing. What I was suggesting with the retry option is since the WW system knows that there is a parsing error and is about to send a WhatsApp message, could it not execute a retry also a few minutes later. Also the timeout error in the log text also says "try again" so why can't the system try again automatically? It just makes sense to me.

Thanks again for replying on a weekend.

duplada

Just had my another parsing failure @ 22:10 I was sat in front of my PC at the time got the WW WhatsApp and rescheduled it for 4 minutes later which executed. So a retry would have succeeded. Unfortunately the most expensive ones (when they fail) run during the IOG period overnight, so even with a WhatsApp notification its not always possible to troubleshoot it, where a retry may well succeed.

admin

duplada We’ll get a retry mechanism in for this.

duplada

Thank you for implementing a retry that can only help matters.

I had a further failure @03:24 the expensive one (4 to 100% charge)

GE must have done something with the API. My schedules have been running like clockwork for over two months and then suddenly all these null errors start happening. Do they update all the time (daily) like WW or do they have specific releases every so often?

What does that API call actually do? Does it read the latest data recorded on the GE server or does it actually query my Gateway in real time?

I am quite prepared to have a go at GE over this problem, can you send me a complete failed response to that API call from my system and I will ask PL to look at it for me.

admin

duplada It's simply returning voltages, grid load, solar generation, battery soc etc.

But it seems in your case, occasionally the call to https://api.givenergy.cloud/v1/inverter/:serial/system-data/latest for your AIO GW just returns this response:

{"data":{"time":null,"status":"Unknown","solar":{"power":null,"arrays":[{"array":1,"voltage":null,"current":null,"power":null},{"array":2,"voltage":null,"current":null,"power":null}]},"grid":{"voltage":null,"current":null,"power":null,"frequency":null},"battery":{"percent":null,"power":null,"temperature":null},"inverter":{"temperature":null,"power":null,"output_voltage":null,"output_frequency":null,"eps_power":null},"consumption":null}}

Basically, it may (and I'm just spitballing here) have something to do with their sampling frequency and caching expiration policies. I'm sure they probably cache it for a minimal amount of time and have the server return this. In any case, there should be no reason why the call for system-data/latest should ever return nulls. If real-time isn't available, it should fallback to the latest cached version of say a few minutes ago or so.

Hopefully with this info, they can diagnose and fix this for you.

duplada

I have reported the issue to GE let's see if I receive a reply. If you can let me know when the retry function is implemented.

admin

duplada The issue will be, what if 3 retries at say 2.5, 5 and 10 seconds still isn't sufficient? And I suspect that could well be the case...

duplada

Oh I forgot and thanks again for all your help with this Great Support.

duplada

Sorry I didn't spot your reply. Retry 3 minutes later if possible?

admin

duplada Too long to wait tbh, needs to be done by the same server function in that case, without baking in other specific mechanisms for just this scenario.

We'll see how we go with the initial retry implementation.

duplada

Will you amend the message if it fails to include the retry count?