<span><ul><li><strong>Role Title:</strong><span></span>Senior Software Engineer – LLM Evaluation &amp; Repository Validation</li><li><strong>Role Type</strong>: Contractor assignment (no medical/paid leave)</li><li><strong>Location</strong>: Remote</li></ul><p><br></p><p><strong>What does day-to-day look like:</strong></p><p>1. Analyze and triage GitHub issues across trending open-source libraries.</p><p>2. Set up and configure code repositories, including Dockerization and environment setup.</p><p>3. Evaluating unit test coverage and quality.</p><p>4. Modify and run codebases locally to assess LLM performance in bug-fixing scenarios.</p><p>5. Collaborate with researchers to design and identify repositories and issues that are challenging for LLMs.</p><p>6. Opportunities to lead a team of junior engineers to collaborate on projects.</p><p><br></p><p><strong>Required Skills:</strong></p><p>1. Strong experience with at least one of the following languages: Python, JavaScript, Java, Go, Rust, C/C++, C#, or Ruby.</p><p>2. Proficiency with Git, Docker, and basic software pipeline setup.</p><p>3. Ability to understand and navigate complex codebases.</p><p>4. Comfortable running, modifying, and testing real-world projects locally.</p><p>5. Experience contributing to or evaluating open-source projects is a plus.</p><p><br></p><p><strong>Nice to Have:</strong></p><p>1. Previous participation in LLM research or evaluation projects.</p><p>2. Experience building or testing developer tools or automation agents.</p><p><br></p><p><strong>Offer details:</strong></p><p>1. Commitment required: 20 hours per week with some overlap with PST</p><p>2. Employment type: Contractor assignment (no medical/paid leave)</p><p>3. Duration of contract: 3 months with expected start date as next week</p><p><br></p><p><strong>What You’ll Do</strong></p><p>1. Analyze and triage GitHub issues across trending open-source libraries.</p><p>2. Set up and configure code repositories, including Dockerization and environment setup.</p><p>3. Evaluating unit test coverage and quality.</p><p>4. Modify and run codebases locally to assess LLM performance in bug-fixing scenarios.</p><p>5. Collaborate with researchers to design and identify repositories and issues that are challenging for LLMs.</p><p><br></p><p><strong>Apply Now!</strong></p><button><span><span><span>…</span><span> more</span></span></span></button></span>