In reply to Greg Brown.

Hi @Frederike

Where exactly is all this specified in privacy policy of wp.org?

The privacy policy does not mention all of these details, it speaks in broader terms about how wordpress.org handles data, privacy, and how third parties are interacted with. Then we adhere to those policies: https://wordpress.org/about/privacy/. I’m giving these details to provide additional transparency since it came up and there was some confusion and details certainly do matter.

What exactly is included in those “about two months of logs”?

Other than what I mentioned above already, we also have things like reasons for the errors, the date and time, how long the request took, the data center it was processed in. The tags I mentioned above are really the only thing I think is at all “interesting” in them.

Mostly I look at the logs in aggregate for:
– How fast we are handling the requests: 50% in less than 20ms and 95% in less than 80ms
– Volume of failures: Of about 7m requests over the past 7 days there were a few hundred queries that failed. Mostly they all look like a few bursts of bad requests that are too small for anyone to have bothered to look into.
– What features of the Elasticsearch API are being used and how often.
– Examining the JSON of a failing search query so I can run it manually and understand why it is failing.

In general it is that aggregate information that is of most use to me and the reason why we keep the logs. When we find something is going wrong it is also useful to look for patterns in the timing of when they occurred. In the past we were only tracking the errors, but for handling ES system upgrades it is helpful to understand what query features are being used, and exactly when errors started occurring.

What is triggering deletion after “about two months”?

It is automatically deleted.

Do you realize that all this has to be public and some even needs user consent due to GDPR all over Europe?

That is what the WordPress Privacy Policy is about. Yes. I’m probably not the best person to discuss with if you think something is missing. Maybe someone in the core-privacy or meta meetings: https://make.wordpress.org/meetings/. Feel free to ping me and pull me in though, happy to be a part of that discussion (@gibrown).