About the Team The ByteDance ByteStore team is responsible for the research, development, operation, and maintenance of ByteStore – a self‑developed distributed storage system. Serving as the underlying global storage platform, we support multiple storage and computing products, including Object Storage, Block Storage, Relational Databases, NoSQL Databases, and Message Queues. We are committed to building a large‑scale, highly reliable, highly available, high‑performance, and low‑cost distributed storage infrastructure to provide superior data storage and access capabilities for applications.
About the Role We are looking for talented, good‑thinking SREs at all levels to join the ByteStore team. In this SRE‑focused role, your work will span reliability engineering, operational automation, system performance profiling, capacity planning, and incident response. You will help ensure that ByteStore meets strict availability, durability, and latency SLOs across global data centers. We prefer candidates who embrace a culture of openness, self‑drive, and strong collaboration in the following areas:
- Design, develop, and maintain SRE tooling – Build automation for deployment, configuration management, failover, disaster recovery drills, and capacity planning to reduce operational toil.
- Participate in deployment and release processes – Lead safe change management (canary, gradual rollout, rollback) for ByteStore components across production environments.
- Research cutting‑edge storage & SRE technologies – Explore new reliability patterns, chaos engineering, observability techniques, and cost‑efficient storage hardware/software.
- Partner with business teams – Continuously improve the stability, functionality, and performance of ByteStore from an operational perspective, ensuring downstream products (databases, queues, object storage) have reliable storage.
- Participate in online operational changes, on‑call duties, and issue investigation – Join on‑call rotations, respond to incidents, drive root cause analysis, and lead post‑mortems to prevent recurrence.
- Independently troubleshoot complex, cross‑layer issues – Diagnose deep system problems spanning storage nodes, networks, clients, and dependencies. Demonstrate cross‑team coordination to resolve large‑scale outages or data integrity issues.
Requirements
Minimum Qualifications
- Bachelor's or higher degree in Computer Science or equivalent.
- Strong coding skills in C++, with a solid code quality awareness (essential for writing reliable automation and debugging core storage components).
- Deep understanding of classical distributed storage systems in the industry (e.g., HDFS) from an operational and performance perspective.
- High levels of troubleshooting and problem‑solving capability
- Excellent teamwork, communication skills, and customer service orientation – ability to work with product teams and developers.
Preferred Qualifications
- Experience with storage tech stacks, including NVMe, SPDK/DPDK, and metadata subsystems.
- Proficiency in unit/functional/system testing, with rigorous implementation habits.
- Capability of troubleshooting and performance tuning on Linux – strong plus.
- Hands‑on experience with large‑scale distributed storage systems in production SRE or operations roles.
- Familiarity with distributed systems, and development or maintenance experience in distributed storage systems (HDFS, Ceph, GlusterFS, etc.) – plus.
- Experience with observability tools (Prometheus, Grafana, ELK), chaos engineering, and capacity planning is highly desirable.