Skip to content
Snippets Groups Projects
  1. Oct 19, 2023
  2. Oct 18, 2023
  3. Oct 17, 2023
    • vlorentz's avatar
      4b371d5d
    • vlorentz's avatar
      e29c98c3
    • vlorentz's avatar
      Prevent timestamps in node properties from being shifted · 86f0c2d0
      vlorentz authored
      according to the timezone WriteNodeProperties is being run in.
      
      Due to our ORC exports using the `timestamp` instead of the `timestamp with timezone`, reader and writer need to agree out of bound on the timezone used in files they exchange.
      
      However, we don't do this:
      
      * `swh-dataset` uses pyorc, which uses the C++ ORC library, which assumes users (us) always write in GMT
      * `swh-graph` uses the Java ORC library, which assumes the system timezone (or `$TZ` if set)
      
      So when reading with a non-UTC timezone, the Java ORC library interprets timestamps in the dataset as being in the local timezone, and converts them to UNIX timestamps (number of seconds since epoch); then we use these converted timestamps and write them to `.property.author_timestamp.bin` and `.property.committer_timestamp.bin`.
      
      This commit regenerates the example graph to have the correct timestamps. It also applies the 39ed0d17 change that removes useless padding at the end of all property files.
      
      Resolves #4788
      86f0c2d0
  4. Sep 30, 2023
  5. Oct 16, 2023
  6. Oct 13, 2023
  7. Oct 12, 2023
  8. Oct 06, 2023
  9. Oct 05, 2023
  10. Oct 04, 2023
  11. Oct 03, 2023
  12. Sep 28, 2023
  13. Sep 27, 2023
  14. Sep 26, 2023
  15. Sep 20, 2023
  16. Sep 13, 2023
  17. Sep 07, 2023
Loading