Table Of Contents

This Page

Diopside Release Notes

Version 5 Diopside

Introduction

This release is based on XRootD V5.

v5.1.13 Diopside

2023-06-03

Bug

  • [EOS-5546] - MGM: IoStat fprintf() stuck
  • [EOS-5555] - FST segfaults around qclient::QSet::srem
  • [EOS-5559] - EOS HTTP REST API - no JSON output if authentication is done with Bearer token

New features

  • [EOS-5561] - Create “eos df” command

v5.1.12 Diopside

2023-02-28

Bug

  • [EOS-5526] - User Sessions count seems to be wrong
  • [EOS-5534] - LRU should not walk down the recycle bin and apply policies
  • [EOS-5535] - LRU tries to delete all directories having an empty deletion policy
  • [EOS-5542] - Error when accessing directories with wildcards

Improvement

  • [EOS-5536] - LRU code has still in-memory namespace code

v5.1.11 Diopside

2023-02-15

Bug

  • [EOS-5516] - Dangling files (possibly) after container is removed
  • [EOS-5520] - eos CLI group resolution changed - INC3372876
  • [EOS-5523] - eosxd recovery failing

Improvement

  • [EOS-5524] - Allow https gateway nodes to provide x-forwarded-for headers

v5.1.10 Diopside

2023-02-07

Note

  • Update dependency to eos-xrootd-5.5.7 which also matches XRootD-5.5.2

Bug

  • [EOS-5386] - iostat reports are not processed fast enough

Improvements

  • MGM: Make central balancer configurable at runtime
  • FST: Chunk fsck requests to at most 50k entries per request
  • MGM: enable hide-version also when heartbrate has been changed

v5.1.9 Diopside

2023-01-24

Bug

  • [EOS-5487] - sticky bit on version folders makes Recycler not able to clean the files on the recycle bin.
  • [EOS-5488] - New Year’s crashes on all projects and homes
  • [EOS-5489] - PropFind fails when namespace mappings should apply
  • [EOS-5494] - eosxd looping when cleaning write queue
  • [EOS-5495] - FST crashing while doing LevelDB->ext_attr conversion on a (not) broken (enough) disk
  • [EOS-5498] - All 0 size files are marked as missing when using xattr fmd

New Feature

  • [EOS-5209] - Fsck removal should just move stripes to a quarantine directory

Improvement

  • [EOS-5501] - Allow black and whitelisting of token vouchers (ids)

v5.1.8 Diopside

2022-12-14

Note

  • Update dependency eos-xrootd-5.5.5
  • Includes an important fix for HTTP TPC PULL transfers.

Bug

  • [EOS-5467] - Inspector aggregates results instead of reseting the current scan
  • MGM: Add regfree in FuseServer regex usage to avoid memory leak
  • MGM: Unlock the Access mutex when delaying a client to not get problems to get a write lock

Improvement

  • [EOS-5478] - Invert Stall logic to check first user limits and then catch-all rules

v5.1.7 Diopside

2022-12-12

Bug

  • [EOS-5474] - Conversion breaks files with FMD info in xattrs

Improvement

  • [EOS-5469] - Allow to select secondary groups with kerberos authentication and implement AC checks for secondorary groups
  • [EOS-5471] - Add atime to EOS
  • [EOS-5458] - Setting a namespace xattr might fail for wopi

v5.1.6 Diopside

2022-12-05

Bug

  • [EOS-5467] - Inspector aggregates results instead of reseting the current scan

Improvement

  • [EOS-5465] - Shoe FUSE application name in ‘fusex ls’
  • [EOS-5466] - Add Stall / NoStall host lists to access interface

v5.1.5 Diopside

2022-12-02

Bug

  • MGM: Fix MGM crash when the balancer is configured

Improvement

  • [EOS-5452] - New metric: Provide I/O errors per transfer in report logs
  • [EOS-5453] - New metric: Namespace contention calculation in ns stat command
  • [EOS-5131] - RFE: honour XRD_APPNAME for xrdcp
  • [EOS-5444] - Provide number of stripes in the inspector command
  • [EOS-5454] - EOS inspector: Provide layout_id in the list output per fxid
  • [EOS-5455] - eos node ls monitoring - Improve sys.uptime value format
  • [EOS-5459] - MGM: avoid blocking cleanup ops while user mapping
  • [EOS-5464] - Have TPC transfers respect the client tpc.ttl value

v5.1.4 Diopside

2022-11-22

Bug

  • [EOS-5442] - eosxd crash (on shutdown) under ShardedCache destructor
  • [EOS-5446] - Failures in setting thread names

v5.1.3 Diopside

2022-11-16

Bug

  • [EOS-5162] - Setting ACL does not work when dir ends with whitespace
  • [EOS-5433] - GroupBalancer: crash when conversions are scheduled before Converter
  • [EOS-5436] - Origin Restriction does not work as expected
  • [EOS-5437] - Fix potential leaks in Mapping::getPhysicalIds

New Feature

  • [EOS-5145] - Extending lock support
  • [EOS-5438] - Don’t stall clients when thread pool is exhausted and a rate limit is reached

Improvement

  • [EOS-5231] - Allow eos attr set to operate on CIDs
  • [EOS-5344] - eos recycle -m: show inode used / max numbers
  • [EOS-5401] - Return the inode number in FMD responses for GRPC
  • [EOS-5412] - add qclient performance metrics on monitoring format.
  • [EOS-5413] - QClient performance: have last 5m, last 1m, etc metrics
  • [EOS-5439] - Add eosxd3 to all builds when fuse3 is available and ship in the RPM

v5.1.2 Diopside

2022-10-04

Bug

  • [EOS-5399] - FST: Segfaults in FmdConverter
  • [EOS-5400] - FST crash in AccountMissing due to null Fmd object

Improvement

  • [EOS-3297] - Print the deviation used for the group balancer

New features

  • MGM: Add implementation for central group balancer using TPC

v5.1.1 Diopside

2022-09-15

Note

  • Update dependency to eos-xrootd-5.5.1
  • eosd is now deprecated and there are no more RPM packages provided for it

Bug

  • [EOS-5347] - EOS token not usable via eosxd

  • [EOS-5369] - Occasional error during eoscta test “mismatch between requested fid/fsid and retrieved ones”

  • [EOS-5371] - Fix crash of the MGM when listing container entries due to invalidated

    iterators to the ContainerMap/FileMap objects.

  • FST: eos-xrootd-5.5.1 fixes a bug in XRootD related to async close functionality where the FST would crash if it received another requests for a file which was in the process of being closed.

New features

  • CTA: Enhance/extend EOS report messages for CTA prepare workflow

v5.1.0 Diopside

2022-09-02

Note

  • This release comes with XRootD/eos-xrootd 5.5.0 as dependency

Bug

  • [EOS-5377] - Unhandled exception in the GeoBalancer code
  • [EOS-5367] - Fix IoStat intialization when there is no prior data in QuarkDB
  • MGM: Fsck: correct the calculation of expected number of stripes in RepairFstXsSzDiff

Improvement

  • [EOS-5380] - Qclient: handle folly warnings
  • [EOS-5381] - Fix potential format overflows
  • [EOS-5378] - Fix compilation warnings
  • FUSEX: Add support for json statistics output

New features

  • FST: Add support for storing file metadata info as extended attributes of the raw files on disk rather than using the LevelDB on disk. Disabled by default for the moment.

v5.0.31 Diopside

2022-08-12

Bug

  • FST: Properly detect HTTP transfers and skip async close functionality in such cases
  • [EOS-5359] - use after free in fusex::client::info
  • [EOS-5358] - WNC GRPC unserialized global options

v5.0.30 Diopside

2022-08-11

Bug

  • [EOS-5355] - System ACLs evaluation overruling logic is incorrect

New Feature

  • [EOS-5342] - CREATE cta workflow not triggered when new file created using fusex - DELETE workflow is also missing

Improvement

  • [EOS-5343] - Better enforcement of the scattered placement policy

v5.0.29 Diopside

2022-07-29

Bug

  • Fix /usr/bin/python dependency on EL8(S) which is no longer provided by any package, therefore we need to explicitly use /usr/bin/python3

v5.0.28 Diopside

2022-07-26

Note

  • This version of EOS is based on an internal release of XRootD namely eos-xrootd-5.4.7

Bug

  • [EOS-5336] - Lot of EOS FST crash (SIGSEGV) in the EOSALICE instance
  • [EOS-5308] - MGM: Potential double free in LDAP initialize
  • [EOS-5334] - LDAP connection socket leak
  • [EOS-5335] - MGM crash in Fileinfo.cc:97

v5.0.27 Diopside

2022-06-30

Bug

  • [EOS-5296] - FST segfault around XrdXrootdProtocol::Process2
  • [EOS-5314] - segfault around “XrdCl::CopyProcess::CleanUpJobs”
  • [EOS-5302] - Iostat domain accounting is broken
  • [EOS-5303] - Shared filesystem file registration feature
  • [EOS-5308] - MGM: Potential double free in LDAP initialize

Improvement

  • [EOS-5317] - Crash in AssignLBHandler with asan
  • [EOS-5321] - Allow to define which errors the fsck repair thread works on
  • [EOS-5305] - Tape REST API - V1 with an option to deactivate STAGE

v5.0.26 Diopside

2022-06-21

Note

  • XRootD: Based on eos-xrootd-5.4.5 which fixes a couple for important bugs on the xrootd client side.

Bug

  • [EOS-5302] - Iostat domain accounting is broken
  • [EOS-5303] - Shared filesystem file registration feature

Improvements

  • MGM: Make fsck start up and shutdown more responsive
  • MGM: Add fsck repair procedure for m_mem_sz_diff errors

v5.0.25 Diopside

2022-06-09

Bug

  • [EOS-5278] - Segmentation fault around eos::mgm::GroupDrainer::scheduleTransfer
  • [EOS-5284] - GroupBalancer: spurious logs when no transfers can be scheduled
  • [EOS-5286] - Physical quota is not updated when we set EC conversion
  • [EOS-5288] - Wrong layout id after conversion operation leading to wrong physical size
  • [EOS-5218] - Infinite loop in XrdCl::XRootDMsgHandler::Copy
  • MGM: The initial behaviour of xrdfs prepare -s/-a/-e and xrdfs query prepare have been restored

Improvement

  • [EOS-5277] - Add LockMonitor class wrapping standard mutex
  • [EOS-5282] - Allow converter configuration to persist on restarts
  • [EOS-5285] - GroupDrainer: Allow all transfers to be reset
  • [EOS-5289] - File truncate can be slow especially for RAIN layouts
  • [EOS-5290] - File close operation for RAIN layouts can trigger client timeouts
  • MGM: Tape REST API v0.1 release - Support for ArchiveInfo and Release functionality + discovery endpoint
  • MISC: Allow the eos-iam-mapfile tool to deal with DNs containing commas

v5.0.24 Diopside

2022-05-27

Bug

  • [EOS-3713] - sys.eos.mdino should not use old-style inodes
  • [EOS-5230] - Keep xattrs when restoring versions
  • [EOS-5269] - Certain FSes not picked up by the group drainer

Improvement

  • [EOS-5263] - groupmod is hard limited to 256 groups
  • [EOS-5267] - Provide timestamp in eos convert list failed errors

v5.0.23 Diopside

2022-05-16

Note

  • This release uses eos-xrootd-5.4.4 which is based on XRootD-5.4.3-rc3.

Bug

  • [EOS-5246] - replica show ‘error_label=none’ while having checksum mismatch.

Improvement

  • [EOS-5184] - Add RedirectCollapse to XrdMgmOfs::Redirect responses
  • [EOS-5198] - Add few log lines to MasterLog

v5.0.22 Diopside

2022-05-06

Improvements

FUSEX: Refactoring async response handling

v5.0.21 Diopside

2022-05-06

Notes

  • Note: this is a scratch build on top of XRootD-5.4.3-RC1 trying to test

a bug fix concerning vector reads * Update dependency to XRootD-5.4.3-RC1

v5.0.20 Diopside

2022-05-03

Improvements

MGM: Improve fsck handling for rain files with rep_diff_n errors MGM: Add extra logging in fsck and be more defensive when handling unregistered stripes MGM: Group drainer prune transfers only once every few minutes FST: Silence stat errors for TPC transfers during preparation stages

v5.0.19 Diopside

2022-05-02

Bug

  • MGM: Fix race condition in Converter which can lead to wrong metadata stored in leveldb for converted files.
  • MGM: Fix wrong computation of number of stripes for RAIN layout
  • [EOS-5199] - Metadata (xattrs) is lost when creating new versions
  • [EOS-5219] - eos fsck report json output does not reflect command line options -l and -i
  • [EOS-5224] - No update is perfomed when adding a new member to an e-group in EOSATLAS

New Feature

  • [EOS-5178] - Implement Group Drain
  • [EOS-5225] - Have a useful GroupDrain Status

v5.0.18 Diopside

2022-04-22

Bug

  • [EOS-5197] - Deleting an xattr via console does not delete the key
  • [EOS-5199] - Metadata (xattrs) is lost when creating new versions
  • MGM: Fix crash in debug message when Env object is null for Access method

New Feature

  • [EOS-5215] - Fsck handle stripe size inconsistencies for RAIN layouts

Improvement

  • [EOS-4955] - Add project quota tests as a part of CI
  • MGM: Iostat performance improvements for summary output
  • MGM: Iostat make extra tables optional by default and add separate flag for displaying them.

v5.0.17 Diopside

2022-04-13

Note

  • This version includes add the fixes up to 4.8.82.

Improvement

  • [EOS-5201] - Allow for more fine grained IO policies
  • [EOS-5204] - Only create files via FUSEX if there is inode and volume quota and physical space available
  • [EOS-5205] - Distinguish writable space and total space
  • [EOS-5206] - Don’t allow to set quota volume lower than the minimum fuse quota booking size

v5.0.16 Diopside

2022-03-29

Bug

  • [EOS-5181] - Slave to Master redirection creates IO errors on FUSEx mounts
  • [EOS-5176] - Make OAuth tolerant to self-signed//invalid certificates used by identity provider

Improvement

  • MGM: Add protection against multi-source retry for RAIN layouts
  • MGM: Rewrite of the IoStat implementation for better accuracy
  • MGM: Remove dependency on eos-scitokens and use the library provided by XRootD framework
  • DOC: Update documentation concerning the MGM configuration for SciTokens support
  • NS: QuarkSyncTimeAccounting - removed namespace lock usage

New feature

  • MGM: Add support for eos tokens over https

v5.0.15 Diopside

2022-03-22

Note

  • Includes all the changes from 4.8.79

Bug

  • FUSEX: never keep the deletion mutex when distroying an upload proxy because the destructor still needs a free call back thread to use HandleResponse
  • [EOS-5153] - EC file written via FUSEx - mismatching checksum
  • [EOS-5167] - MGM segv in a non-tape enabled instance

v5.0.14 Diopside

2022-03-14

Bug

  • [EOS-5090] - convert clear is not a admin command
  • [EOS-5133] - node ls -b does not remove the domain names
  • FUSEX: Fix deadlocks and race-conditions reported by TSAN

Improvement

  • [EOS-5108] - workaround: drop forced automount expiry on FUSEX updates
  • [EOS-5126] - [eos-ns-inspect] Complement stripediff ouput

v5.0.13 Diopside

2022-02-15

Note

  • Includes all the changes from 4.8.76

Bug

  • [EOS-5110] - Consolidate Access control in GRPC MD, MDSTreaming
  • [EOS-5116] - Workaround for XrdOucBuffPool bug
  • [EOS-5118] - eos-ns-inspect scan is initializing maxdepth to 0, even if not used
  • [EOS-5119] - Deadlock scenario in eosxd

Improvement

  • [EOS-5111] - Groupbalancer: newly introduced fields may not have a sane value
  • [EOS-5120] - io stat tag totals

v5.0.12 Diopside

2022-02-04

Note

  • Identical to 5.0.11 but re-tagged due to Koji issues

v5.0.11 Diopside

2022-02-04

Bug

  • [EOS-5105] - eosxd crash in cap::quotax::dump

v5.0.10 Diopside

2022-02-02

Note

  • This release includes all the changes from 4.8.74 release

Bug

  • [EOS-5069] - filesystem status in “rw + failed”
  • [EOS-5070] - Access::ThreadLimit creates re-entrant lock of the access mutex
  • [EOS-5095] - Re-entrant lock triggered by out of quota warning

Improvement

  • [EOS-5065] - Add create-if-not-exists option in GRPC
  • [EOS-5076] - Extend iotype interfaces to be space/directory defined
  • MGM: Fix missing support for cid/cxid and error output for convert command
  • WNC: Replaced auxiliary ACL function for fileinfo command

New features

  • WNC: Implemented support for EOS-wnc token, convert, fsck and new find commands
  • WNC: Changed GRPC streaming mechanism for find, ls and transfer commands

v5.0.9 Diopside

2022-01-12

Bug

  • COMMON: Avoid segv due to mutex object set to nullptr in RWLock printout
  • [EOS-4850] - eosxd crash in destructor under metad::pmap::retrieveWithParentTS()
  • [EOS-5057] - Volume quota dispatched to FUSE clients mixes logical and physical bytes

v5.0.8 Diopside

2022-01-06

Note

  • Note: This release includes all the changes to the 4.8.70 release

Bug

  • [EOS-5039] - Threads with parens in their name cannot access EOS

Improvement

  • [EOS-5029] - Allow to apply rate limiting in recursive (server side) command.
  • [EOS-5048] - Support direct IO for high performance read/write use cases

v5.0.7 Diopside

2021-12-01

Note

  • Release based on XRootD-5.3.4

New features

  • WNC: Implemeneted support for EOS-wnc member, backup, map and archive command

v5.0.6 Diopside

2021-11-16

Note

  • Release based on XRootD-5.3.3 which fixes a critical bug concerning “invalid responses”

Bug

  • ARCHIVE: Avoid trying to set extended attributes which are empty
  • [EOS-4995] MGM/CONSOLE: add ‘-c’ option to CLI ls to show also the checksum for a listing
  • CTA: Fixed FST crash when connecting to misconfigured ctafrontend endpoint

v5.0.5 Diopside

2021-11-04

Bug

OSS: Avoid leaking file descriptors for xsmap files which are deleted in the meantime MGM: Skip applying fsck config changes at the slave as these will be properly

v5.0.4 Diopside

2021-10-27

Bug

  • SPEC: Make sure both libproto* and libXrd* requirements are excluded when building the eos packages since these come from internally build rpms like eos-xrootd and eos-protobuf3 which don’t expose the library so names so that they can be installed on a machine along with the official rpms for the corresponding packages if they exist.
  • MGM: Avoid that a slave MGM applies an fsck configuration change in a loop

Improvements

  • EOS-4967: Add ARM64 support for blake3

v5.0.3 Diopside

2021-10-27

Note

  • This version is based on XRootD 5.3.2 that addresses some critical bug observed in the previous version for XRootD.

Bug

  • MGM: Fix GRPC IPv6 parsing
  • [EOS-4963] - FST: Reply with 206(PARTIAL_CONTENT) for partial content responses
  • [EOS-4962] - MGM: Return FORBIDDEN if there is a public access restriction in PROFIND requests
  • [EOS-4950] - FUSEX: fix race conditions in async callbacks with respect to proxy object deletions

New features

  • [EOS-4670] - FUSEX: implement file obfuscation and encryption

v5.0.2 Diopside

2021-09-06

Bug

  • [EOS-4809] - Make eos5 work with XrdMacaroons from XRootD5
  • Includes all the fixes from 4.8.65

Improvements

  • WNC: Improvements to the EOS-Drive for fileinfo & health command

v5.0.1 Diopside

2021-08-16

New features

  • Comtrade WNC contribution for the server side
  • Includes all the fixes from the 4.8.60 release

v5.0.0 Diopside

2021-06-11

Major changes

  • Based on XRootD 5.2.0
  • Drop support for in-memory namespace
  • Drop support for file based configuration
  • Drop support for old high-availability setup
  • Make fusex classes compatible with the latest protobuf library
  • Integrate QuarkDB as part of the eos release process