Skip to content
Snippets Groups Projects
  1. Nov 30, 2023
  2. Nov 29, 2023
  3. Nov 27, 2023
  4. Nov 20, 2023
  5. Nov 15, 2023
  6. Oct 31, 2023
  7. Oct 23, 2023
    • vlorentz's avatar
      Prevent timestamps in node properties from being shifted according to the... · 38f66160
      vlorentz authored
      Prevent timestamps in node properties from being shifted according to the timezone WriteNodeProperties is being run in.
      
      Due to our ORC exports using the `timestamp` instead of the `timestamp with timezone`, reader and writer need to agree out of bound on the timezone used in files they exchange.
      
      However, we don't do this:
      
      * `swh-dataset` uses pyorc, which uses the C++ ORC library, which assumes users (us) always write in GMT
      * `swh-graph` uses the Java ORC library, which assumes the system timezone (or `$TZ` if set)
      
      So when reading with a non-UTC timezone, the Java ORC library interprets timestamps in the dataset as being in the local timezone, and converts them to UNIX timestamps (number of seconds since epoch); then we use these converted timestamps and write them to `.property.author_timestamp.bin` and `.property.committer_timestamp.bin`.
      
      This commit regenerates the example graph to have the correct timestamps. It also applies the 39ed0d17 change that removes useless padding at the end of all property files.
      
      Resolves #4788
      38f66160
  8. Oct 16, 2023
  9. Aug 21, 2023
  10. Aug 14, 2023
  11. Aug 12, 2023
  12. Aug 07, 2023
  13. Aug 03, 2023
  14. Aug 02, 2023
  15. Jul 20, 2023
  16. Jul 13, 2023
  17. Jul 12, 2023
  18. Jul 11, 2023
  19. Jul 10, 2023
  20. Jul 07, 2023
  21. Jun 28, 2023
  22. Jun 27, 2023
Loading