- At least 3-5 years of industry experience in the following:
- Big Data development (Spark, Scala, Java)
- Hadoop platform (Solr search & indexing, HBase, HDFS) Data Streaming (Kafka, Flume)
- RESTful web services (JAX-RS, Jersey/Spring)
- Full Stack web development & deployment on a RHEL platform
- WebSphere, Tomcat
- Test-driven design
Your Tasks – Paint the world green
- Holistic cloud-based infrastructure automation
- Distributed data processing clusters as well as data streaming platforms based on Kafka, Flink and Spark
- Microservice platforms based on Docker
- Development infrastructure and QA automation
- Continuous Integration/Delivery/Deployment
Your Profile – Ready to hop on board
- Experience in building and operating complex infrastructure
- Expert-level: Linux, System Administration
- Experience with Cloud Services, Expert-Level with either AWS or GCP
- Experience server and operation-system-level virtualization is a strong plus, in particular practical experience with Docker and cluster technologies like Kubernetes, AWS ECS, OpenShift
- Mindset: "Automate Everything", "Infrastructure as Code", "Pipelines as Code", "Everything as Code"
- Hands-on experience with "Infrastructure as Code" tools: TerraForm, CloudFormation, Packer
- Experience with a provisioning / configuration management tools (Ansible, Chef, Puppet, Salt)
- Experience designing, building and integrating systems for instrumentation, metrics/log collection, and monitoring: CloudWatch, Prometheus, Grafana, DataDog, ELK
- At least basic knowledge in designing and implementing Service Level Agreements
- Solid knowledge of Network and general Security Engineering
- At least basic experience with systems and approaches for Test, Build and Deployment automation (CI/CD): Jenkins, TravisCI, Bamboo
- At least basic hands-on DBA experience, experience with data backup and recovery
- Experience with JVM-based build automation is a plus: Maven, Gradle, Nexus, JFrog Artifactory
Dich sprechen Agilität, Product driven IT, Cloud Computing und Machine Learning an?
Du bist leistungsorientiert und hast den Mut Neues auszuprobieren?
Wir haben den digitalen Wandel in unserer DNA verankert!
- Das Aufgabenset umfasst vor allem Engineering (IBM MQ Linux, z/OS) und Betrieb von Middleware-Komponenten (File Transfer, Web Service Infrastruktur).
- Im Detail heisst das Komponentenverantwortung (u.A. Lifecycling, Zur Verfügungstellung von API's und Self-Services, Automatisierung der Abläufe, Erstellung und Pflege der Dokumentation), Sicherstellung des Betriebs (Du ergreifst autonom die notwendigen Massnahmen, Bereitschaft zu sporadischen Wochenendeinsätzen/Pikett), als auch Wissenspflege und -vermittlung.
- In einem agilen Umfeld, mithilfe bei der Migration unserer Komponenten in die Cloud.
Deine Fähigkeiten und Talente:
- Du bringst ein abgeschlossenes Informatikstudium oder vergleichbare Erfahrung mit.
- Dein Know-How umfasst Messaging Middleware-Komponenten, idealerweise IBM MQ auf Linux angereichert mit z/OS Knowhow, cool wären zudem noch Kenntnisse von RabbitMQ und Kafka.
- Andere Middleware Komponenten (File Transfer und Web Service) sind Dir nicht gänzlich unbekannt und Übertragungsprotokolle als auch die Linux-Welt im Speziellen sind Dir vertraut.
- Du bringst fundierte Erfahrung in der Automatisierung an den Tisch (Bash, Python) und auch REST, API's sowie Java(-script) sind keine Fremdwörter für Dich. Erste Programmiererfahrung in einer objektorientierten Sprache, vorzugsweise Java, runden dein Profil ab.
- Du bist integrativ, betrachtest Herausforderungen aus verschiedenen Perspektiven und stellst unbequeme Fragen, wenn es darauf ankommt.
- Du bist sicher in der deutschen und englischen Sprache.
Job Title: Tableau Engineer
Duration: 6-12 Months+ (potential to go perm)
Location: Atlanta, GA (30328) - Onsite
Notes from Manager:
We need a data analyst who knows Tableau, scripting (JSON, Python), Altreyx API, AWS, Analytics.
The Tableau Software engineer will be a key resource to work across our Software Engineering BI/Analytics stack to ensure stability, scalability, and the delivery of valuable BI & Analytics solutions for our leadership teams and business partners. Keys to this position are the ability to excel in identification of problems or analytic gaps and mapping and implementing pragmatic solutions. An excellent blend of analytical, technical and communication skills in a team based environment are essential for this role.
· Able to design, build, maintain & deploy complex reports in Tableau
· Experience integrating Tableau into another application or native platforms is a plus
· Expertise in Data Visualization including effective communication, appropriate chart types, and best practices.
· Knowledge of best practices and experience optimizing Tableau for performance.
· Experience reverse engineering and revising Tableau Workbooks created by other developers.
· Understand basic statistical routines (mean, percentiles, significance, correlations) with ability to apply in data analysis
· Able to turn ideas into creative & statistically sound decision support solutions
Education and Experience:
· Bachelors degree in Computer Science or equivalent work experience
· 3-5 Years of hands on experience in data warehousing & BI technologies (Tableau/OBIEE/Business Objects/Cognos)
· Three or more years of experience in developing reports in Tableau
· Have good understanding of Tableau architecture, design, development and end user experience.
What We Look For:
· Very proficient in working with large Databases in Oracle & Big Data technologies will be a plus.
· Deep understanding & working experience of data warehouse and data mart concepts.
· Understanding of Alteryx and R packages is a plus
· Experience designing and implementing high volume data processing pipelines, using tools such as Spark and Kafka.
· Experience with Scala, Java or Python and a working knowledge of AWS technologies such as GLUE, EMR, Kinesis and Redshift preferred.
· Excellent knowledge with Amazon AWS technologies, with a focus on highly scalable cloud-native architectural patterns, especially EMR, Kinesis, and Redshift
· Experience with software development tools and build systems such as Jenkins
The Senior Data Engineer focuses on designing, implementing and supporting new and existing data solutions- data processing, and data sets to support various advanced analytical needs. You will be designing, building and supporting data pipelines consuming data from multiple different source systems and transforming it into valuable and insightful information. You will have the opportunity to contribute to end-to-end platform design for our cloud architecture and work multi-functionally with operations, data science and the business segments to build batch and real-time data solutions. The role will be part of a team supporting our Corporate, Sales, Marketing, and Consumer business lines.
MINIMUM QUALIFICATIONS AND REQUIREMENTS:
- 7+ years of relevant experience in one of the following areas: Data engineering, business intelligence or business analytics
- 5-7 years of supporting a large data platform and data pipelining
- 5+ years of experience in scripting languages like Python etc.
- 5+ years of experience with AWS services including S3, Redshift, EMR andRDS
- 5+ years of experience with Big Data Technologies (Hadoop, Hive, HBase, Pig, Spark, etc.)
- Expertise in database design and architectural principles and methodologies
- Experienced in Physical data modeling
- Experienced in Logical data modeling
- Technical expertise should include data models, database design and data mining
PRINCIPAL DUTIES AND RESPONSIBILITIES:
- Design, implement, and support a platform providing access to large datasets
- Create unified enterprise data models for analytics and reporting
- Design and build robust and scalable data integration (ETL) pipelines using SQL, Python, and Spark.
- As part of an Agile development team contribute to architecture, tools and development process improvements
- Work in close collaboration with product management, peer system and software engineering teams to clarify requirements and translate them into robust, scalable, operable solutions that work well within the overall data architecture
- Coordinate data models, data dictionaries, and other database documentation across multiple applications
- Leads design reviews of data deliverables such as models, data flows, and data quality assessments
- Promotes data modeling standardization, defines and drives adoption of the standards
- Work with Data Management to establish governance processes around metadata to ensure an integrated definition of data for enterprise information, and to ensure the accuracy, validity, and reusability of metadata
Got a taste for something new?
We’re Grubhub, the nation’s leading online and mobile food ordering company. Since 2004 we’ve been connecting hungry diners to the local restaurants they love. We’re moving eating forward with no signs of slowing down.
With more than 90,000 restaurants and over 15.6 million diners across 1,700 U.S. cities and London, we’re delivering like never before. Incredible tech is our bread and butter, but amazing people are our secret ingredient. Rigorously analytical and customer-obsessed, our employees develop the fresh ideas and brilliant programs that keep our brands going and growing.
Long story short, keeping our people happy, challenged and well-fed is priority one. Interested? Let’s talk. We’re eager to show you what we bring to the table.
About the Opportunity:
Senior Site Reliability Engineers are embedded in Big Data specific Dev teams to focus on the operational aspects of our services, and our SREs run their respective products and services from conception to continuous operation. We're looking for engineers who want to be a part of developing infrastructure software, maintaining it and scaling. If you enjoy focusing on reliability, performance, capacity planning, and the automation everything, you’d probably like this position.
Some Challenges You’ll Tackle
TOOLS OUR SRE TEAM WORKS WITH:
- Python – our primary infrastructure language
- Docker (in production!)
- Splunk, Spark, Hadoop, and PrestoDB
- Python and Fabric for automation and our CD pipeline
- Jenkins for builds and task execution
- Linux (CentOS and Ubuntu)
- DataDog for metrics and alerting
You Should Have
- Experience in AWS services like Kinesis, IAM, EMR, Redshift, and S3
- Experience managing Linux systems
- Configuration management tool experiences like Puppet, Chef, or Ansible
- Continuous integration, testing, and deployment using Git, Jenkins, Jenkins DSL
- Exceptional communication and troubleshooting skills.
NICE TO HAVE:
- Python or Java / Scala development experience
- Bonus points for deploying/operating large-ish Hadoop clusters in AWS/GCP and use of EMR, DC/OS, Dataproc.
- Experience in Streaming data platforms, (Spark streaming, Kafka)
- Experience developing solutions leveraging Docker
Are you passionate about data architecture? Are you interested in shaping the next generation of data science driven products for the financial industry? Do you enjoy working in an agile environment involving multiple stakeholders?
Responsible for selecting appropriate technologies from open source, commercial on-premises and cloud-based offerings. Integrating a new generation of tools within the existing environment to ensure access to accurate and current data. Consider not only the functional requirements, but also the non-functional attributes of platform quality such as security, usability, and stability.
We want you to help us to strengthen and further develop the transformation of Avaloq to a data driven product company. Make analytics scalable and accelerate the process of data science innovation.
- PhD, Master or Bachelor degree in Computer Science, Math, Physics, Engineering, Statistics or other technical field
- Knowledgeable with BigData technologies and architectures (e.g. Hadoop, Spark, data lakes, stream processing)
- Practical experience with Container Platforms (OpenShift) and/or containerization software (Kubernetes, Dockers)
- Hands-on experience developing data extraction and transformation pipelines (ETL process)
- Expert knowledge in RDBMS, NoSQL and Data Warehousing
- Familiar with information retrieval software such as Elastic Search/Lucene/SOLR
- Firm understanding of major programming/scripting languages like Java/Scala, Linux, PHP, Python and/or R
- High integrity, responsibility and confidentiality a requirement for dealing with sensitive data
- Strong presentation and communication skills
- Good planning and organisational skills
- Collaborative mindset to sharing ideas and finding solutions
- Fluent in English; German, Italian and French a plus
- Be a thought leader for best practice how to develop and deploy data science products & services
- Provide an infrastructure to make data driven insights scalable and agile
- Liaise and coordinate with stakeholders regarding setting up and running a BigData and analytics platform
- Lead the evaluation of business and technical requirements
- Support data-driven activities and a data-driven mindset where needed
Main place of work
Avaloq Evolution AG
Anna Drozdowska, Talent Acquisition Professional
Allmendstrasse 140 - 8027 Zürich - Switzerland
Please only apply online.
Note to Agencies: All unsolicited résumés will be considered direct applicants and no referral fee will be acknowledged.
As part of our Data Business Group, you will lead technology innovation for our clients through robust delivery of world-class solutions. You will build better software better! There will never be a typical day and thats why people love it here. The opportunities to make a difference within exciting client initiatives are unlimited in the ever-changing technology landscape. You will be part of a highly collaborative and growing network of technology and data experts, who are taking on todays biggest, most complex business challenges using the latest data and analytics technologies. We will nurture your talent in an inclusive culture that values diversity. You will have an opportunity to work in roles such as Data Scientist, Data Engineer, or Chief Data Officer covering all aspects of Data including Data Management, Data Governance, Data Intelligence, Knowledge Graphs, and IoT. Come grow your career in Technology at Accenture!
People in our Client Delivery & Operations career track drive delivery and capability excellence through the design, development and/or delivery of a solution, service, capability or offering. They grow into delivery-focused roles, and can progress within their current role, laterally or upward.
Business & Technology Integration professionals advise upon, design, develop and/or deliver technology solutions that support best practice business changes
The Bus&Industry Integration Assoc Mgr aligning technology with business strategy and goals they working directly with the client gathering requirements to analyze, design and/or implement technology best practice business changes. They are sought out as experts internally and externally for their deep functional or industry expertise, domain knowledge, or offering expertise. They enhance Accenture's marketplace reputation.
Data and Analytics professionals define strategies, develop and deliver solutions that enable the collection, processing and management of information from one or more sources, and the subsequent delivery of information to audiences in support of key business processes.
Data Management professionals define strategies and develop/deliver solutions and processes for managing enterprise-wide data throughout the data lifecycle from capture to processing to usage across all layers of the application architecture.
A professional at this position level within Accenture has the following responsibilities:
Identifies, assesses and solves complex business problems for area of responsibility, where analysis of situations or data requires an in-depth evaluation of variable factors.
Closely follows the strategic direction set by senior management when establishing near term goals.
Interacts with senior management at a client and/or within Accentureon matters where they may need to gain acceptance on an alternate approach.
Has some latitude in decision-making. Acts independently to determine methods and procedures on new assignments.
Decisions have a major day to day impact on area of responsibility.
Manages large - medium sized teams and/or work efforts (if in an individual contributor role) at a client or within Accenture.
- Minimum of 3 plus years of hands-on technical experience implementing Big Data solutions utilizing Hadoop or other Data Science and Analytics platforms.
- Minimum of 3 plus years of experience with a full life cycle development from functional design to deployment
- Minimum 2 plus years of hands-on technical experience with delivering Big Data Solutions in the cloud with AWS or Azure
- Minimum 3 plus years of hands-on technical experience in developing solutions utilizing at least two of the following:
- Kafka based streaming services
- R Studio
- Cassandra , MongoDB
- MapReduce, Pig, Hive
- Scala, Spark
- knowledge on Jenkins, Chef, Puppet
- Bachelor's degree or equivalent years of work experience
- Ability to travel 100%, Monday- Thursday
- Proven ability to build, manage and foster a team-oriented environment
- Proven ability to work creatively and analytically in a problem-solving environment
- Desire to work in an information systems environment
- Excellent communication (written and oral) and interpersonal skills
- Excellent leadership and management skills
Applicants for employment in the US must have work authorization that does not now or in the future require sponsorship of a visa for employment authorization in the United States and with Accenture.
Candidates who are currently employed by a client of Accenture or an affiliated Accenture business may not be eligible for consideration.
Accenture is a federal contractor and an EEO and Affirmative Action Employer of Females/Minorities/Veterans/Individuals with Disabilities.
Equal Employment Opportunity
All employment decisions shall be made without regard to age, race, creed, color, religion, sex, national origin, ancestry, disability status, veteran status, sexual orientation, gender identity or expression, genetic information, marital status, citizenship status or any other basis as protected by federal, state, or local law.
Job candidates will not be obligated to disclose sealed or expunged records of conviction or arrest as part of the hiring process.
The Sr. Engineer will be building pipelines using Spark ScalaMust Haves:
Expertise in the Big Data processing and ETL Pipeline
Designing large scaling ETL pipelines - batch and realtime
Expertise in Spark Scala coding and Data Frame API (rather than the SQL based APIs)
Expertise in core Data Frame APIs
Expertise in doing unit testing Spark Data frame API based code
Strong in Scripting knowledge using Python and shell scripting
Experience and expertise in working on performance tuning of large scale data pipelines
- Be an integral member on the team responsible for design, implement and maintain distributed big data capable system with high-quality components (Kafka, EMR + Spark, Akka, etc).
- Embrace the challenge of dealing with big data on a daily basis (Kafka, RDS, Redshift, S3, Athena, Hadoop/HBase), perform data ETL, and build tools for proper data ingestion from multiple data sources.
- Collaborate closely with data infrastructure engineers and data analysts across different teams, find bottlenecks and solve the problem
- Design, implement and maintain the heterogeneous data processing platform to automate the execution and management of data-related jobs and pipelines
- Implement automated data workflow in collaboration with data analysts, continue to improve, maintain and improve system in line with growth
- Collaborate with Software Engineers on application events, and ensuring right data can be extracted
- Contribute to resources management for computation and capacity planning
- Diving deep into code and constantly innovating
- Experience with AWS data technologies (EC2, EMR, S3, Redshift, ECS, Data Pipeline, etc) and infrastructure.
- Working knowledge in big data frameworks such as Apache Spark, Kafka, Zookeeper, Hadoop, Flink, Storm, etc
- Rich experience with Linux and database systems
- Experience with relational and NoSQL database, query optimization, and data modelling
- Familiar with one or more of the following: Scala/Java, SQL, Python, Shell, Golang, R, etc
- Experience with container technologies (Docker, k8s), Agile development, DevOps and CI tools.
- Excellent problem-solving skills
- Excellent verbal and written communication skills
ConocoPhillips is the worlds largest independent E&P company based on production and proved reserves. Headquartered in Houston, Texas, ConocoPhillips had operations and activities in 17 countries, $71 billion of total assets, and approximately 11,100 employees as of Sept. 30, 2018. Production excluding Libya averaged 1,221 MBOED for the nine months ended Sept. 30, 2018, and proved reserves were 5.0 billion BOE as of Dec. 31, 2017.
Employees across the globe focus on fulfilling our core SPIRIT Values of safety, people, integrity, responsibility, innovation and teamwork. And we apply the characteristics that define leadership excellence in how we engage each other, collaborate with our teams, and drive the business.
The Sr. Analytics Analyst will be part of the Production, Drilling, and Projects Analytics Services Team within the Analytics Innovation Center of Excellence that enables data analytics across the ConocoPhillips global enterprise. This role works with business units and global functions to help strategically design, implement, and support data analytics solutions. This is a full-time position that provides tremendous career growth potential within ConocoPhillips.
Responsibilities May Include
- Complete end to end delivery of data analytics solutions to the end user
- Interacting closely with both business and developers while gathering requirements, designing, testing, implementing and supporting solutions
- Gather business and technical specifications to support analytic, report and database development
- Collect, analyze and translate user requirements into effective solutions
- Build report and analytic prototypes based on initial business requirements
- Provide status on the issues and progress of key business projects
- Providing regular reporting on the performance of data analytics solutions
- Delivering regular updates and maintenance on data analytics solutions
- Championing the data analytics solutions and technologies at ConocoPhillips
- Integrate data for data models used by the customers
- Deliver Data Visualizations used for data driven decision making
- Provide strategic technology direction while supporting the needs of the business
- Legally authorized to work in the United States
- 5+ years of related IT experience
- 5+ year of Structure Querying Language experience (ANSI SQL, T-SQL, PL/SQL)
- 3+ years hands-on experience delivering solutions with an Analytics Tools i.e. (Spotfire, SSRS, Power BI, Tableau, Business Objects)
- Bachelor's Degree in Information Technology or Computer Science
- 5+ years of Oil and Gas Industry experience
- 5+ years hands-on experience delivering solutions with Informatica PowerCenter
- 5+ years architecting data warehouses and/or data lakes
- 5+ years with Extract Transform and Load (ETL) tools and best practices
- 3+ years hands-on experience delivering solutions with Teradata
- 1+ years developing analytics models with R or Python
- 1+ years developing visualizations using R or Python
- Experience with Oracle (11g, 12c) and SQL Server (2008 R2, 2010, 2016) and Teradata 15.x
- Experience with Hadoop technologies (Hortonworks, Cloudera, SQOOP, Flume, etc.)
- Experience with AWS technologies (S3, SageMaker, Athena, EMR, Redshift, Glue, etc.)
- Thorough understanding of BI/DW concepts, proficient in SQL, and data modeling
- Familiarity with ETL tools (Informatica, etc.) and ETL processes
- Solutions oriented individual; learn quickly, understand complex problems, and apply useful solutions
- Ability to work in a fast-paced environment independently with the customer
- Ability to work as a team player
- Ability to work with business and technology users to define and gather reporting and analytics requirements
- Strong analytical, troubleshooting, and problem-solving skills experience in analyzing and understanding business/technology system architectures, databases, and client applications to recognize, isolate, and resolve problems
- Demonstrates the desire and ability to learn and utilize new technologies in data analytics solutions
- Strong communication and presentation skills
- Takes ownership of actions and follows through on commitments by courageously dealing with important problems, holding others accountable, and standing up for what is right
- Delivers results through realistic planning to accomplish goals
- Generates effective solutions based on available information and makes timely decisions that are safe and ethical
Candidates for this U.S. position must be a U.S. citizen or national, or an alien admitted as permanent resident, refugee, asylee or temporary resident under 8 U.S.C. 1160(a) or 1255(a) (1). Individuals with temporary visas such as A, B, C, D, E, F, G, H, I, J, L, M, NATO, O, P, Q, R or TN or who need sponsorship for work authorization in the United States now or in the future, are not eligible for hire.
ConocoPhillips is an Equal Opportunity/Affirmative Action Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, national origin, age, disability, veteran status, gender identity or expression, genetic information or any other legally protected status.
Information Management-Information Technology
Individual Contributor/Staff Level
Line of Business
Feb 13, 2019, 4:56:49 PM
Sr. Data Engineer Hadoop, Spark, Data Pipelines, Growing Company
One of our clients is looking for a Sr. Data Engineer in the Fort Worth, TX area! Build your data expertise with projects centering on large Data Warehouses and new data models! Think outside the box to solve challenging problems! Thrive in the variety of technologies you will use in this role!
Why should I apply here?
- Culture built on creativity and respect for engineering expertise
- Nominated as one of the Best Places to Work in DFW
- Entrepreneurial environment, growing portfolio and revenue stream
- One of the fastest growing mid-size tech companies in DFW
- Executive management with past successes in building firms
- Leader of its technology niche, setting the standards
- A robust, fast-paced work environment
- Great technical challenges for top-notch engineers
- Potential for career growth, emphasis on work/life balance
- A remodeled office with a bistro, lounge, and foosball
What will I be doing?
- Building data expertise and owning data quality for the transfer pipelines that you create to transform and move data to the companys large Data Warehouse
- Architecting, constructing, and launching new data models that provide intuitive analytics to customers
- Designing and developing new systems and tools to enable clients to optimize and track advertising campaigns
- Using your expert skills across a number of platforms and tools such as Ruby, SQL, Linux shell scripting, Git, and Chef
- Working across multiple teams in high visibility roles and owning the solution end-to-end
- Providing support for existing production systems
- Broadly influencing the companys clients and internal analysts
What skills/experiences do I need?
- B.S. or M.S. degree in Computer Science or a related technical field
- 5+ years of experience working with Hadoop and Spark
- 5+ years of experience with Python or Ruby development
- 5+ years of experience with efficient SQL (Postgres, Vertica, Oracle, etc.)
- 5+ years of experience building and supporting applications on Linux-based systems
- Background in engineering Spark data pipelines
- Understanding of distributed systems
What will make my résumé stand out?
- Ability to customize an ETL or ELT
- Experience building an actual data warehouse schema
Location: Fort Worth, TX
Citizenship: U.S. citizens and those authorized to work in the U.S. are encouraged to apply. This company is currently unable to provide sponsorship (e.g., H1B).
Salary: 115 130k + 401k Match
Signify Health is looking for a savvy Data Engineer to join our growing team of deep learning specialists. This position would be responsible for evolving and optimizing data and data pipeline architectures, as well as, optimizing data flow and collection for cross-functional teams. The Data Engineer will support software developers, database architects, data analysts, and data scientists. The ideal candidate would be self-directed, passionate about optimizing data, and comfortable supporting the Data Wrangling needs of multiple teams, systems and products.
If you enjoy providing expert level IT technical services, including the direction, evaluation, selection, configuration, implementation, and integration of new and existing technologies and tools while working closely with IT team members, data scientists, and data engineers to build our next generation of AI-driven solutions, we will give you the opportunity to grow personally and professionally in a dynamic environment. Our projects are built on cooperation and teamwork, and you will find yourself working together with other talented, passionate and dedicated team member, all working towards a shared goal.
Essential Job Responsibilities:
- Assemble large, complex data sets that meet functional / non-functional business requirements
- Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing data models for greater scalability, etc.
- Leverage Azure for extraction, transformation, and loading of data from a wide variety of data sources in support of AI/ML Initiatives
- Design and implement high performance data pipelines for distributed systems and data analytics for deep learning teams
- Create tool-chains for analytics and data scientist team members that assist them in building and optimizing AI workflows
- Work with data and machine learning experts to strive for greater functionality in our data and model life cycle management capabilities
- Communicate results and ideas to key decision makers in a concise manner
- Comply with applicable legal requirements, standards, policies and procedures including, but not limited to the Compliance requirements and HIPAA.
- High school diploma or equivalent.
- Bachelors degree in Computer Science, Electrical Engineer, Statistics, Informatics, Information Systems, or another quantitative field. or related field or equivalent work experience.
- 5+ years of experience in a Data Engineer role.
- Experience using the following software/tools preferred:
- Experience with big data tools: Hadoop, Spark, Kafka, etc.
- Experience with relational SQL and NoSQL databases, including Postgres and Cassandra.
- Experience with AWS or Azure cloud services.
- Experience with stream-processing systems: Storm, Spark-Streaming, etc.
- Experience with object-oriented/object function scripting languages: Python, Java, C#, etc.
- Strong work ethic, able to work both collaboratively, and independently without a lot of direct supervision, and solid problem-solving skills
- Must have strong communication skills (written and verbal), and possess good one-on-one interpersonal skills.
- Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases
- A successful history of manipulating, processing and extracting value from large disconnected datasets.
- Working knowledge of message queuing, stream processing, and highly scalable big data data stores.
- 2 years of experience in data modeling, ETL development, and Data warehousing
- Fluently speak, read, and write English
- Fantastic motivator and leader of teams with a demonstrated track record of mentoring and developing staff members
- Strong point of view on who to hire and why
- Passion for solving complex system and data challenges and desire to thrive in a constantly innovating and changing environment
- Excellent interpersonal skills, including teamwork and negotiation
- Excellent leadership skills
- Superior analytical abilities, problem solving skills, technical judgment, risk assessment abilities and negotiation skills
- Proven ability to prioritize and multi-task
- Advanced skills in MS Office
- In Leadership Do whats right, even if its tough
- In Collaboration Leverage our collective genius, be a team
- In Transparency Be real
- In Accountability Recognize that if it is to be, its up to me
- In Passion Show commitment in heart and mind
- In Advocacy Earn trust and business
- In Quality Ensure what we do, we do well
- Fast-paced environment
- Requires working at a desk and use of a telephone and computer
- Normal sight and hearing ability
- Use office equipment and machinery effectively
- Ability to ambulate to various parts of the building
- Ability to bend, stoop
- Work effectively with frequent interruptions
- May require occasional overtime to meet project deadlines
- Lifting requirements of
Are you interested in scalability & distributed systems? Do you want to work to help shaping a discovery engine powered by cutting edge technologies and machine learning at scale? If you answered yes to the above questions, Mix's Research and Development is the team for you!
In this role, you'll be part of a small and innovative team comprised of engineers and data scientists working together to understand content by leveraging machine learning and NLP technologies. You will have the opportunity to work on core problems like detection of low quality content or spam, text semantic analysis, video and image processing, content quality assessment and monitoring. Our code operates at massive scale, ingesting, processing and indexing millions of URLs.
- Write code to build an infrastructure, which is capable of scaling based on the load
- Collaborate with researchers and data scientists to integrate innovative Machine Learning and NLP techniques with our serving, cloud and data infrastructure
- Automate build and deployment process, and setup monitoring and alerting systems
- Participate in the engineering life-cycle, including writing documentation and conducting code reviews
- Strong knowledge of algorithms, data structures, object oriented programming and distributed systems
- Fluency in OO programming language, such as Scala (preferred), Java, C, C++
- 3+ years demonstrated expertise in stream processing platforms like Apache Flink, Apache Storm and Apache Kafka
- 2+ years experience with a cloud platform like Amazon Web Services (AWS) or Microsoft Azure
- 2+ years experience with monitoring frameworks, and analyzing production platforms, UNIX servers and mission critical systems with alerting and self-healing systems
- Creative thinker and self-starter
- Strong communication skills
- Experience with Hadoop, Hive, Spark or other MapReduce solutions
- Knowledge of statistics or machine learning
Comcast's Technology & Product organization works at the intersection of media and technology. Our innovative teams are continually developing and delivering products that transform the customer experience. From creating apps like TVGo to new features such as the Talking Guide on the X1 platform, we work every day to make a positive impact through innovation in the pursuit of building amazing products that are enjoyable, easy to use and accessible across all platforms. The team also develops and supports our evolving network architecture, including next-generation consumer systems and technologies, infrastructure and engineering, network integration and management tools, and technical standards.
Software engineering skills combined with the demands of a high volume, highly-visible analytics platform make this an exciting challenge for the right candidate.
Are you passionate about digital media, entertainment, and software services? Do you like big challenges and working within a highly motivated team environment?
As a software engineer in the Data Experience (DX) team, you will research, develop, support, and deploy solutions in real-time distributing computing architectures. The DX big data team is a fast-moving team of world-class experts who are innovating in providing user-driven, self-service tools for making sense and making decisions with high volumes of data. We are a team that thrives on big challenges, results, quality, and agility.
Who does the data engineer work with?
Big Data software engineering is a diverse collection of professionals who work with a variety of teams ranging from other software engineering teams whose software integrates with analytics services, service delivery engineers who provide support for our product, testers, operational stakeholders with all manner of information needs, and executives who rely on big data for data backed decisioning.
What are some interesting problems you'll be working on?
Develop systems capable of processing millions of events per second and multi-billions of events per day, providing both a real-time and historical view into the operation of our wide-array of systems. Design collection and enrichment system components for quality, timeliness, scale and reliability. Work on high-performance real-time data stores and a massive historical data store using best-of-breed and industry-leading technology.
Where can you make an impact?
Comcast DX is building the core components needed to drive the next generation of data platforms and data processing capability. Running this infrastructure, identifying trouble spots, and optimizing the overall user experience is a challenge that can only be met with a robust big data architecture capable of providing insights that would otherwise be drowned in an ocean of data.
Success in this role is best enabled by a broad mix of skills and interests ranging from traditional distributed systems software engineering prowess to the multidisciplinary field of data science.
- Develop solutions to big data problems utilizing common tools found in the ecosystem.
- Develop solutions to real-time and offline event collecting from various systems.
- Develop, maintain, and perform analysis within a real-time architecture supporting large amounts of data from various sources.
- Analyze massive amounts of data and help drive prototype ideas for new tools and products.
- Design, build and support APIs and services that are exposed to other internal teams
- Employ rigorous continuous delivery practices managed under an agile software development approach
- Ensure a quality transition to production and solid production operation of the software
Skills & Requirements:
- 5+ years programming experience
- Bachelors or Masters in Computer Science, Statistics or related discipline
- Experience in software development of large-scale distributed systems including proven track record of delivering backend systems that participate in a complex ecosystem.
- Experience working on big data platforms in the cloud or on traditional Hadoop platforms
- AWS Core
- Step Functions
- API Gateway
- Batch, Streaming, ML
- Performance tuning at scale
- Real-time Streaming
- Data File Formats:
- Avro, Parquet, JSON, ORC, CSV, XML
- NoSQL / SQL
- Microservice development
- RESTful API development
- CI/CD pipelines
- Jenkins / GoCD
- Docker / Kubernetes
- Presto / Athena
- Test-driven development/test automation, continuous integration, and deployment automation
- Enjoy working with data data analysis, data quality, reporting, and visualization
- Good communicator, able to analyze and clearly articulate complex issues and technologies understandably and engagingly.
- Great design and problem solving skills, with a strong bias for architecting at scale.
- Adaptable, proactive and willing to take ownership.
- Keen attention to detail and high level of commitment.
- Good understanding in any: advanced mathematics, statistics, and probability.
- Experience working in agile/iterative development and delivery environments. Comfort in working in such an environment. Requirements change quickly and our team needs to constantly adapt to moving targets.
About Comcast DX (Data Experience):
Data Experience(DX) is a results-driven, data platform research and engineering team responsible for the delivery of multi-tenant data infrastructure and platforms necessary to support our data-driven culture and organization. The mission of DX is to gather, organize, make sense of Comcast data, and make it universally accessible to empower, enable, and transform Comcast into an insight-driven organization. Members of the DX team define and leverage industry best practices, work on extremely large-scale data problems, design and develop resilient and highly robust distributed data organizing and processing systems and pipelines as well as research, engineer, and apply data science and machine intelligence disciplines
Comcast is an EOE/Veterans/Disabled/LGBT employer
Common Purpose, Uncommon
Opportunity. Everyone at Visa works with one goal in mind making sure that Visa is the best way to pay and be paid, for everyone everywhere. This is our global vision and the common purpose that unites the entire Visa team. As a global payments technology company, tech is at the heart of what we do: Our VisaNet network processes over 13,000 transactions per second for people and businesses around the world, enabling them to use digital currency instead of cash and checks. We are also global advocates for financial inclusion, working with partners around the world to help those who lack access to financial services join the global economy. Visas sponsorships, including the Olympics and FIFA World Cup, celebrate teamwork, diversity, and excellence throughout the world. If you have a passion to make a difference in the lives of people around the
world, Visa offers an uncommon opportunity to build a strong, thriving career. Visa is fueled by our team of talented employees who continuously raise the bar on delivering the convenience and security of digital currency to people all over the world. Join our team and find out how Visa is everywhere you want to
The ideal candidate will be responsible for the following to:
- Perform Hadoop Administration on Production Hadoop clusters
- Perform Tuning and Increase Operational efficiency on a continuous basis
- Monitor health of the platforms and Generate Performance Reports and Monitor and provide continuous improvements
- Working closely with development, engineering and operation teams, jointly work on key deliverables ensuring production scalability and stability
- Develop and enhance platform best practices
- Ensure the Hadoop platform can effectively meet performance & SLA requirements
- Responsible for support of Hadoop Production environment which includes Hive, YARN, Spark, Impala, Kafka, SOLR, Oozie, Sentry, Encryption, Hbase, etc.
- Perform optimization capacity planning of a large multi-tenant cluster
- Minimum 3 years of work experience in maintaining, optimization, issue resolution of Hadoop clusters, supporting Business users and Batch
- Experience in Configuring and setting up Hadoop clusters and provide support for - aggregation, lookup & fact table creation criteria
- Map Reduce tuning, data node, NN recovery etc.
- Experience in Linux / Unix OS Services, Administration, Shell, awk scripting
- Experience in building and scalable Hadoop applications
- Experience in Core Java, Hadoop (Map Reduce, Hive, Pig, HDFS, H-catalog, Zookeeper and OOzie)
- Hands-on Experience in SQL (Oracle ) and No SQL Databases (HBASE/Cassandra/Mongo DB)
- Excellent oral and written communication and presentation skills, analytical and problem solving skills
- Self-driven, Ability to work independently and as part of a team with proven track record developing and launching products at scale
- Minimum of four year technical degree required
- Experience on Cloudera distribution preferred
- Hands-on Experience as a Linux Sys Admin is a plus
- Knowledge on Spark and Kafka is a plus.
All your information will be kept confidential according to EEO guidelines.
Job Number: REF15232V
Google Cloud Solutions Architect (Pre Sales)United States | Canada | Remote | Work from Home
Are you a US or Canada based Cloud Solutions Architect who likes to operate with a high degree of autonomy and have diverse responsibilities that require strong leadership, deep technology skills and a dedication to customer service? Do you have Big data and Data centric skills? Do you want to take part in the strategic planning of organizations data estate with a focus of fulfilling business requirements around cost, scalability and flexibility of the platform? Can you draft technology roadmaps and document best practice gaps with precise steps of how to get there? Can you implement the details of the backlogs you have helped build? Do you demonstrate consistent best practices and deliver strong customer satisfaction? Do you enjoy pre sales? Can you demonstrate adoption of new technologies and frameworks through the development of proofs of concepts?
If you have a passion for solving complex problems and for pre sales then this could be the job for you!
What Will You Be Doing?
- Collaborating with and supporting Pythian sales teams in the pre-sales & account management process from the technical perspective, remotely and on-site (approx 75%).
- Defining solutions for current and future customers that efficiently address their needs. Leading through example and influence, as a master of applying technology solutions to solve business problems.
- Developing Proof of Concepts (PoC) in order to demonstrate feasibility and value to Pythians customers (approx 25%).
- Defining solutions for current and future customers that efficiently address their needs.
- Identifying then executing solutions with a commitment to excellent customer service
- Collaborating with others in refining solutions presented to customers
- Conducting technical audits of existing architectures (Infrastructure, Performance, Security, Scalability and more) document best practices and recommendations
- Providing component or site-wide performance optimizations and capacity planning
- Recommending best practices & improvements to current operational processes
- Communicating status and planning activities to customers and team members
- Participate in periodic overtime (occasionally on short notice) travelling up to approx 50%).
What Do We Need From You?
While we realise you might not have everything on the list to be the successful candidate for the Solutions Architect job you will likely have at least 10 years experience in a variety of positions in IT. The position requires specialized knowledge and experience in performing the following:
- Undergraduate degree in computer science, computer engineering, information technology or related field or relevant experience.
- Systems design experience
- Understanding and experience with Cloud architectures specifically: Google Cloud Platform (GCP) or Microsoft Azure
- In-depth knowledge of popular database and data warehouse technologies from Microsoft, Amazon and/or Google (Big Data & Conventional RDBMS), Microsoft Azure SQL Data Warehouse, Teradata, Redshift, BigQuery, Snowflake etc.
- Be fluent in a few languages, preferably Java and Python, and having familiarity with Scala and Go would be a plus.
- Proficient in SQL. (Experience with Hive and Impala would be great)
- Proven ability to work with software engineering teams and understand complex development systems, environments and patterns.
- Experience presenting to high level executives (VPs, C Suite)
- This is a North American based opportunity and it is preferred that the candidate live on the West Coast, ideally in San Francisco or the Silicon Valley area but strong candidates may be considered from anywhere in the US or Canada.
- Ability to travel and work across North America frequently (occasionally on short notice) up to 50% with some international travel also expected.
- Experience Architecting Big Data platforms using Apache Hadoop, Cloudera, Hortonworks and MapR distributions.
- Knowledge of real-time Hadoop query engines like Dremel, Cloudera Impala, Facebook Presto or Berkley Spark/Shark.
- Experience with BI platforms, reporting tools, data visualization products, ETL engines.
- Experience with any MPP (Oracle Exadata/DW, Teradata, Netezza, etc)
- Understanding of continuous delivery and deployment patterns and tools (Jenkins, Artifactory, Maven, etc)
- Prior experience working as/with Machine Learning Engineers, Data Engineers, or Data Scientists.
- A certification such as Google Cloud Professional Cloud Architect, Google Professional Data Engineer or related AWS Certified Solutions Architect / Big Data or Microsoft Azure Architect
- Experience or strong interest in people management, in a player-coach style of leadership longer term would be great.
What Do You Get in Return?
- Competitive total rewards package
- Flexible work environment: Why commute? Work remotely from your home, theres no daily travel requirement to the office!
- Outstanding people: Collaborate with the industrys top minds.
- Substantial training allowance: Hone your skills or learn new ones; participate in professional development days, attend conferences, become certified, whatever you like!
- Amazing time off: Start with a minimum 3 weeks vacation, 7 sick days, and 2 professional development days!
- Office Allowance: A device of your choice and personalise your work environment!
- Fun, fun, fun: Blog during work hours; take a day off and volunteer for your favorite charity.
Job Overview: SoftClouds is looking for a Data Engineer to join our analytics platform team in designing and developing the next generation data and analytics solutions. The candidate should have deep technical skills as well as the ability to understand data and analytics, and an openness to working with disparate platforms, data sources and data formats.
Roles and Responsibilities:
- Experience with MySQL, MS SQL Server, or Hadoop, or MongoDB.
- Writing SQL Queries, tables joins.
- AWS, python, or bash shell scripting
- Have some experience pulling data from Hadoop.
- Analyze data, system and data flows and develop effective ways to store and present data in BI applications
- ETL experience a plus.
- Work with data from disparate environments including Hadoop, MongoDB Talend, and other SQL and NoSQL data stores
- Help develop the next generation analytics platform
- Proactively ensure data integrity and focus on continuous performance improvements of existing processes.
Required skills and experience:
- 5 or more years of experience in software development
- 3 year of experience in writing Data applications using Spark
- Experience in Java and Python
- Familiarity with Agile development methodology `
- Experience with Scala is a plus
- Experience with NoSQL databases, e.g., Cassandra is a plus
- Expertise in Apache Spark & Hadoop.
- Expertise in machine learning algorithms
Education / Experience:
- Bachelor's Degree in Engineering or Computer Science or related field required.
U.S. Citizens/GC/GC EAD are encouraged to apply. We are unable to sponsor at this time. NO C2C or third-party agencies.
Applied Resource Group is seeking a talented and experienced Data Engineer for our client, an emerging leader in the transit solutions space. As an experienced Data Engineer on the Data Services team, you will lead the design, development and maintenance of comprehensible data pipelines and distributed systems for data extraction, analysis, transformation, modelling and visualization. They're looking for independent thinkers that are passionate about technology and building solutions that continually improve the customer experience. Excellent communication skills and the ability to work collaboratively with teams is critical.
- Building a unified data services platform from scratch, leveraging the most suitable Big Data tools following technical requirements and needs
- Exploring and working with cutting edge data processing technologies
- Work with distributed, scalable cloud-based technologies
- Collaborating with a talented team of Software Engineers working on product development
- Designing and delivering BI solutions to meet a wide range of reporting needs across the organization
- Providing and maintaining up to date documentation to enable a clear outline of solutions
- Managing task lists and communicating updates to stakeholders and team members following Agile Scrum methodology
- Working as a key member of the core team to support the timely and efficient delivery of critical data solutions
- Experience with AWS technologies are desired, especially those used for Data Analytics, including some of these: EMR, Glue, Data Pipelines, Lambda, Redshift, Athena, Kinesis, Elasticache, Aurora
- Minimum of 5 years working in developing and building data solutions
- Experience as an ETL/Data warehouse developer with knowledge in design, development and delivery of end-to-end data integration processes
- Deep understanding of data storage technologies for structured and unstructured data
- Background in programming and knowledge of programming languages such as Java, Scala, Node.js, Python.
- Familiarity with cloud services (AWS, Azure, Google Cloud)
- Experience using Linux as a primary development environment
- Knowledge of Big data systems - Hadoop, pig, hive, shark/spark etc. a big plus.
- Knowledge of BI platforms such as Tableau, Jaspersoft etc.
- Strong communication and analytical skills
- Capable of working independently under the direction of the Head of Data Services
- Excellent communication, analytical and problem-solving skills
- Ability to initially take direction and then work on own initiative
- Experience working in AGILE
Nice-to-have experience and skills:
- Masters in Computer-Science, Computer Engineering or equivalent
- Building data pipelines to perform real-time data processing using Spark Streaming and Kafka, or similar technologies.
Ripple is the world’s only enterprise blockchain solution for global payments. Today the world sends more than $155 trillion* across borders. Yet, the underlying infrastructure is dated and flawed. Ripple connects banks, payment providers, corporates and digital asset exchanges via RippleNet to provide one frictionless experience to send money globally.
Ripple is growing rapidly and we are looking for a results-oriented and passionate Senior Software Engineer, Data to help build and maintain infrastructure and empower the data-driven culture of the company. Ripple’s distributed financial technology outperforms today’s banking infrastructure by driving down costs, increasing processing speeds and delivering end-to-end visibility into payment fees, timing, and delivery.
WHAT YOU’LL DO:
- Support our externally-facing data APIs and applications built on top of them
- Build systems and services that abstract the engines and will allow the users to focus on business and application logic via higher-level programming models
- Build data pipelines and tools to keep pace with the growth of our data and its consumers
- Identify and analyze requirements and use cases from multiple internal teams (including finance, compliance, analytics, data science, and engineering); work with other technical leads to design solutions for the requirements
WHAT WE’RE LOOKING FOR:
- Deep experience with distributed systems, distributed data stores, data pipelines, and other tools in cloud services environments (e.g AWS, GCP)
- Experience with distributed processing compute engines like Hadoop, Spark, and/or GCP data ecosystems (BigTable, BigQuery, Pub/Sub)
- Experience with stream processing frameworks such as Kafka, Beam, Storm, Flink, Spark streaming
- Experience building scalable backend services and data pipelines
- Proficient in Python, Java, or Go
- Able to support Node.js in production
- Familiarity with Unix-like operating systems
- Experience with database internals, database design, SQL and database programming
- Familiarity with distributed ledger technology concepts and financial transaction/trading data
- You have a passion for working with great peers and motivating teams to reach their potential