Comcast's Technology & Product organization works at the intersection of media and technology. Our innovative teams are continually developing and delivering products that transform the customer experience. From creating apps like TVGo to new features such as the Talking Guide on the X1 platform, we work every day to make a positive impact through innovation in the pursuit of building amazing products that are enjoyable, easy to use and accessible across all platforms. The team also develops and supports our evolving network architecture, including next-generation consumer systems and technologies, infrastructure and engineering, network integration and management tools, and technical standards.
Softwareengineering leadership and data science skills, combined with the demands of a highly-visible enterprise metadata repository, make this an exciting challenge for the right candidate.
Are you passionate about digital media, entertainment, and software services? Do you likebigchallenges and working within ahighly-motivatedteam environment?
As aSenior Manager in the Metadata Engineeringgroup of the Data Experience (DX) team at Comcast, you will drive the development,deployment, and support of large-scale Metadata platforms using real-time distributed computing architectures.You willalso employ your skills to promote positive changes in our work culture and practices that will improve our productivity, ingenuity, agility, and software development maturity.TheDXdatateam is afast-moving team of world-class experts who are innovating in end-to-end data delivery and analytics. We are a team that thrives onbigchallenges, results, quality, and agility.
Who will you be working with?
DX MetadataEngineering is a diverse collection of professionals who work with a variety of teams ranging from: other software engineering teams whose Metadata repositories integrate with the Centralized Metadata Repository, Portal engineers who develop a UI to support data discovery, software engineers on other DX platforms that ingest, transform, and retrieve data whose metadata is stored in the Centralized Metadata Store, data stewards/data architects who collect and disseminate metadata information,and users who rely on Metadata for data discovery.
What are some interesting problems you'll be working on?
You will manage the design and development of a metadata and business glossary collection and, enrich the system that allows real-time update of the enterprise and satellite metadata repositories using best-of-breed and industry-leading technologies. These repositories contain metadata and lineage for a widely diverse and ever-growing complement of datasets (e.g., Hortonworks, AWS S3, Streaming Data (Kafka/kinesis), streaming data transformation, ML pipeline, Teradata, and RDBMS's). You will lead the design and development of cross-domain cross-platform lineage tooling by using advanced statistical methods and Machine Intelligence algorithms. You will manage development tools to discover data across disparate metadata repositories, develop tools for data governance, and implement processes to rationalize data across the repositories.
Where can you make an impact?
The dx Teamis building the enterprise metadata repository needed to drive the next generation ofdata platforms and data processing capabilities.Building data products, identifying trouble spots, and optimizing the overall user experience is a challenge that can only be met with a robustmetadata repository capable of providing insightsinto the data and its lineage.
Success in this role is best enabled by a broad mix of skills and interests ranging from traditional distributed systems software engineering prowess to the multidisciplinary field of data science.
-Responsible for managing all metadata assets, applications, and supporting processes.
-Closely work with Architects, Product Owners and Solution Engineers to understand product requirements, understand architectural recommendations, and work with solution engineers to develop a viable solution.
-Guide the Metadata Engineering team in identifying product and technical requirements.
-Ensure products and projects are delivered as a roadmap within the agreed budget and time.
-Serve as primary point of contact and liaison between Metadata Engineering and other teams.
-Closely monitor metadata eco-system to ensure that each metadata asset is performing as per SLA and continuously delivering business value.
-Responsible for preparing team budgets, roadmaps, and operational objectives and ensuring operational plans are aligned with business objectives.
-Responsible for selection and recruitment of resources and work to ensure a high-quality stream of candidates in our talent pipeline
-Experience in hiring and managing teams.
-Experienced in managing projects with competing priorities
-Ensure that direct reports keep current with technological developments within the industry.
-Monitor and evaluate competitive applications and products.
-Drive a culture of continuous improvement and innovation
-Pro-actively work to mitigate risks to performance and delivery of our teams
-Promote solutions for integrating metadata and data quality processes into agile methodologies
-Promote blameless post-mortems and ensure that all post-mortem activities are acted upon
Here are some ofthespecific technologies we use:
-Metadata Repositories-Apache Atlas and Informatica MDM
-Spark(AWS EMR, Databricks)
-Kafka, AWS Kinesis
-AWS Glue, AWS Lambda
-Cassandra, RDBMS, Teradata, AWS DynamoDB
-Elasticsearch, Solr, Logstash, Kibana
-Java, Scala, Go, Python, R
-Git,Maven, Gradle, Jenkins
-Puppet, Docker, Terraform, Ansible, AWS CloudFormation
-Hadoop (HDFS, YARN, ZooKeeper, Hive), Presto
Skills & Requirements:
-7+ years of people leadership experience in a software development environment
-2-4 years of experience in metadata-related projects
-Bachelors or Masters inComputer Science, Statisticsor related discipline
-Demonstrated experience in data management; Certified Data Management Professional (CDMP)-Nice to have
-Experience in software development of large-scale distributed systems including a proven track record of delivering backend systems that participate in a complex ecosystem.
-Experienceinmetadata-related open source frameworks preferred
-Experience in using and contributing to Open Source software preferred.
-Partner withdata analysis, data quality, and reporting teams in establishing the best standards and principles around metadata management
-Excellentcommunicator, able to analyze and articulate complex issues and technologies understandably and engagingly
-Great design and problem-solving skills
-Adaptable, proactive and program ownership
-Keen attention to detail and high level of commitment
-Thrivesin a fast-paced agile environment. Requirements change quickly, and our team needs to constantly adapt to moving targets
-A team player with excellent networking, collaborating and influencing skills
About ComcastDX(Data Experience):
dx(Data Experience) is a results-driven, data platform research and engineering team responsible for the delivery of multi-tenant data infrastructure and platforms necessary to support our data-driven culture and organization. We have an overarching objective to gather, organize, and make sense of Comcast data with the intention to reveal business and operational insight, discover actionable intelligence, enable experimentation, empower users, and delight our stakeholders. Members of the dx team define and leverage industry best practices, work on extremely large-scale data problems, design and develop resilient and highly robust distributed data organizing and processing systems and pipelines as well as research, engineer, and apply data science and machine intelligence disciplines.
Our mission is to enable many diverse users with the tools and information to gather, organize, make sense of Comcast data, and make it universally accessible to empower, enable, and transform Comcast into an insight-driven organization.
Comcast is an EOE/Veterans/Disabled/LGBT employer