PyTexas 2023

We discuss two different methodologies we have used to add type annotations to our Python services: gradually adding type annotations, or incrementally re-writing the code with type annotations. We'll discuss the benefits and challenges of each, when either may be appropriate, and our real-world experiences doing both.

Do you have a large Python code-base that you think would benefit from type annotations and a static type checker? Are you wondering how to go about adding those type annotations? Do you think you should just chuck it all and start over? We've been there, and we can help! After a period of rapid product development, we found ourselves with two decently sized code bases that were becoming increasingly difficult to maintain; having some experience with static type checking, we felt confident that it would help improve the developer experience, but we weren't sure how to go about it. In the end, we chose two different approaches for the two projects: the first, we started incrementally re-writing the application code with type annotations from the beginning, and the second, we started slowly adding type annotations to the existing code. Both techniques have their pros and cons, and, in the end, we felt like each was the right decision for that particular project. This talk will discuss the characteristics of each project which led us to chose one method over the other, and the techniques we used to accomplish the task. We'll also discuss how we broke down the work, maintained the type checker configurations and the overall success of the efforts, so far.

When it comes to machine learning projects, verifying and trusting model performance results is a particularly grueling challenge. This talk will explore both how we can use Python to instill confidence in model metrics and the best way to keep models versioned to increase transparency and accessibility across the team. The tactics demonstrated will help developers save precious development time and increase transparency by incorporating metric tracking early on.

Machine learning projects are challenging, but verifying and trusting model performance results is a particularly grueling challenge. The problem becomes more complex with more team members and larger scope. Engineers need to integrate model performance tracking throughout the project lifecycle to ensure that their model results are reliable, reproducible and transparent. This talk will explore both how we can use Python to instill confidence in model metrics and the best way to keep models versioned to increase transparency and accessibility across the team. In this talk, I'll present the following three MLFlow features that can improve transparency and credibility in the model results. If engineers aren’t applying these techniques, they could risk presenting false or irreproducible model results and waste precious development time retracing their steps to find the best model version. *MLFlow Autologging: This feature requires minimal code and stores all relevant configurations, parameters and metrics, reducing the likelihood that some aspect of the model has not been captured. It makes it easy to compare the results of experiments and determine the final model. *MLFlow System Tags: A small but mighty feature, these reserved tags store important metadata. They capture the state of the codebase at the time of the run, such as git branch, commit hash and developer name. It helps solve a familiar issue to machine learning engineers - how to rerun the exact training pipeline code that was used to generate the best metrics in an experiment by another developer. *MLFlow Model Registry: As the project gets closer to finish, it can be daunting to sift through experiments and runs. Leveraging some of MLFlow's Python API, we'll show how engineers can version their best models and transition models to different stages. This makes it easy to share model performance and encourage other engineers to reproduce the results before pushing the model to production. By the end of this talk, Python developers will know how to leverage some powerful techniques to increase model performance credibility, integrate tracking to enhance collaboration over the project lifecycle and version models to always be ready for production.

Looking at open-source security concerns from a socio-technical perspective is currently underexplored. In this talk, we will learn about analyzing such risks by looking at governance structures, pull requests, and GitHub configurations in open-source 3D printer projects and the overall top Python repositories.

3D printers are a critical component in the industry supply chain, so it is important to secure 3D printer software and firmware. However, there are many open-source repositories that are related to 3D printing, meaning that anyone can contribute to a given project; in fact, even some proprietary software was forked from open-source repositories. In this session, we'll talk briefly about security risks of open-source development in the context of 3D printing; in addition, we will look at findings from the overall top 100 Python repositories on GitHub.

There’s more to Python than the language. And there’s more to Build Engineering than building code. An entire ecosystem of amazing tools sit just beyond the atmosphere of Python’s interpreter. It’s a Build Engineer’s job to ensure you’re using the right ones, configured the right way, and that they run cohesively. As IBM Watson Orders' Build Engineer I am the guardian of the developer experience of almost 100 engineers and data scientists in a Python monorepo. In the past year pre-submit time plummeted, the ease-of-use of local development tools skyrocketed, and I became a maintainer of an Open Source Build System. I am restlessly passionate about their developer experience as well as yours. You should be too. Come find out why Build Engineering is important to you and your org, through one person’s constant struggle to expect better, how to find and foster Build Engineering, and how you can build better Python for yourself and others.

Build Engineering is a labor of love whose benefits can outstretch beyond one project or organization. Allow me to share with you what Build Engineering is, why it is important to you, and how we all stand on the shoulders of giants. #### What is Build Engineering? Build Engineering encompasses more than the instructions we give computers to assemble and deploy our code. It is the incorporation of all the processes/workflows that the humans use to build software. In that sense, Build Engineering is responsible for the team's Developer Experience: the experience of the interaction between the human and the software. Build Engineering doesn't have to come from a Build Engineer either. Although sufficiently large organizations hire dedicated Build Engineers, Build Egnineering can come from anyone motivated enough to care. #### Why should I care about Build Engineer/Developer Experience? The truth is every developer is having an experience with their tools (or more accurately multiple experiences, but let's simplify). Some have it better than others. Good tooling can lead to fast turnaround times, higher code confidence, and happy devs. Bad tooling can result in slow, unhappy devs working in a difficult codebase. #### Standing on the Shoulders of Giants Want to jumpstart your own team's Build Engineering? Come find out the mix of tools I've blended for the developers at IBM Watson Orders, with a focus on speed, cohesion, and simplicity. What else exists in the Python-verse and how they could boost your team? And finally, where we as an ecosystem have room for improvement.

An introduction to recursion and the Python-specific functionality for recursion. Recursion is not just for computer science wizards, and this talk breaks it down into smaller (and smaller!) concepts.

Recursion has an intimidating reputation for being the advanced skill of coding sorcerers. But in this tutorial we look behind the curtain of this formidable technique to discover the simple ideas under it. Through live coding demos in the interactive shell, we'll answer the following questions: * What is recursion, and when is it a good idea and bad idea to use it? * What's a stack, the call stack, and a stack overflow? * What are all the confusing ways that recursion is commonly taught? * Do some problems require recursion? Can recursion do anything a loop can't? * What is memoization, and how does functools.lru_cache work? * How do I draw that cool-looking recursive fractal artwork with Python's turtle module? Beginners will be able to follow this talk. All that is required is a willingness to learn, and a willingness to have the willingness to learn, and... so on.

Interested in machine learning and chatbots? This talk is for you! We'll go over the OpenAI API and use cases, particularly how you can build your own ChatGPT over text message using Python.

ChatGPT and OpenAI are taking us by storm. This talk will dive into OpenAI's API and its many models, methods, and use cases, particularly how you can build your own ChatGPT over text message using Python and Twilio.

Public Cloud is often touted as a cheaper option, but the bills we get in the mail can be long and convoluted. For better or worse, any dollar spent in the cloud is a result of an architecture decision. This talk will give attendees a firm understanding of the relationship between the architecture and the invoice and the key metrics to keep an eye on. We will cover: * What is FinOps and why is it important * Key metrics for measuring Cloud spend * Automating Cloud Invoices Using Python, a SQL database, and FaaS, we can build a pipeline that automatically retrieves our invoice, and populates a searchable table to derive insights such as our top 5 most used services and biggest growing services. Attendees with small developers account to large multi-account deployments will better understand the impact of their architecture on their wallet.

Cloud Economics or FinOps focuses on the efficiency of cloud architecture in performance per cost. It's a key topic for start-ups as they position themselves for growth and fundraising. It's important for engineers to understand how much thier resources actually cost, even as a solo developer. Engineering Team Leads and Architects will get the most out of this talk as we will go over how to derive key metrics that can provide context to Finance Teams. We will use functions-as-a-service and a python runtime to fetch a cloud invoice, import that data into a SQL database, and use that to get Month-over-Month and Year-over-Year reporting.

As real-time event streaming has become a necessity in many organizations, a plethora of libraries, frameworks, and applications have arisen to help meet that need. These different tools each have their own strengths and weaknesses, and you may end up deciding to use one or more of them, but many problems can be solved with a simpler approach.

In this session, we will look at several techniques that we can use to build real-time applications with just Apache Kafka and the confluent-kafka Python package. You may be surprised at how far we can go with these simple tools, but we will also discuss the challenges you might face and the signs that you may need to pull a more powerful tool out of your toolbox. To avoid being too abstract, we’ll put some of our new knowledge to work by building an application that processes real-time social media data to see who is doing the most talking about #PyTexas. Attendees will be able to join in the conversation and maybe see their names on the leaderboard!

Do you find it harder to work with parts of your project? Do you shy away from working with aspects of your code? When showing people what you're working on, do you feel the need to explain why it got that way? As our projects grow, they usually become harder to maintain. This happens when we focus too much on adding new features. As a result, our code becomes hard to change or understand. This talk will give you a tool you can use to unwind that code, making it easier to change and understand, not to mention giving you more satisfaction.

"By continuously improving the design of code, we make it easier and easier to work with. This is in sharp contrast to what typically happens: little refactoring and a great deal of attention paid to expediently adding new features. If you get into the hygienic habit of refactoring continuously, you'll find that it is easier to extend and maintain code." — Joshua Kerievsky, Refactoring to Patterns Refactoring is a vital skill for keeping projects healthy and growing. Without it, implementing new features and maintaining your project becomes painful. But most developers don’t refactor their code. Moreover, many aren’t even sure what it means to refactor. Even worse yet, following traditional refactoring advice can make your Python code worse. Thankfully, a better way has emerged in the last few years. This talk will walk you through how to refactor any piece of code.

Show some examples of useful decorators and talk about the pros and cons of the decorator protocol.

Decorators are a fascinating piece of syntactic sugar in the Python language. These powerful bits of code can be used to reduce boilerplate code and make it easier to move around features. Let’s explore a few simple examples of decorators and talk about the pros and cons of decorators as a feature.

Healthcare organizations generate and store vast amounts of data, but accessing and utilizing this data can be a challenge. One solution to this problem is FHIR (Fast Healthcare Interoperability Resources), a standard for exchanging electronic health information. In this talk, we will explore the basics of FHIR and its data model, as well as how to use Python to interact with FHIR servers and manipulate FHIR data. We will also discuss real-world examples of how FHIR and Python are being used in the healthcare industry to improve patient care and drive innovation. Attendees will come away with a solid understanding of FHIR and Python, and be equipped to start using these technologies to unlock the power of their own health data.

n/a

Do you ever look at your software systems and workflows holistically? Do you ever wonder what could be achieved through smarter designs and dedicated effort? When you design your applications, do you consider developer efficiency and satisfaction as much as end-user experience? Python offers the simplicity and flexibility to achieve both. In this talk, I will explore a journey from supporting disjointed components and addressing ad hoc business requests to developing a technical vision and creating an elegant solution that both serves business needs and offers developers a pleasant support experience. I will also highlight the design thinking approach, software architecture “-ilities” concepts, and the decision process that took place during this project.

As software developers, it is not uncommon having to address unexpected changes, be it emergency requests, changing requirements, or a newly discovered bug. Often, there will be a quick and straightforward solution that seems simple enough to warrant a quick change. The path of least resistance might be tempting, especially when we are pressed on time. But, is it always the best long-term solution? Will it result in duplicate code or introduce hidden hard-coded values? As we gain experience and domain knowledge, we may foresee similar situations arising repeatedly in the future. But, the shortcomings of such an approach are not hard to miss: a growing code base that must be supported, a newly-found bug that will have to be addressed everywhere, retesting and redeployment. The same goes for new enhancements that must be introduced everywhere, while maintaining the same overhead. And what about developer satisfaction and efficiency? How happy are we to maintain this code and how likely are we to introduce new bugs as the requirements grow and the code expands? As importantly, how long will it take us to accommodate a new requirement? At one point during the pandemic, I was faced with similar dilemmas. I work in the field of regulatory technologies. Screening for sanctions, financial violations, money laundering, etc., is one of the most critical activities in this domain. With the tightening regulatory space and expanding business, we knew we would time and time again need to add new screening actions. So, how did we address our users’ needs in a timely manner, while, at the same time, saving our own time and effort? This was a start of an exciting journey, which led me from these questions through the design process to the implementation of the system, which keeps proving itself as an efficient and extensible solution for the regulatory needs we face. After this talk, my audience will gain an understanding of design thinking principles, software “-ilities” and, hopefully, develop their own ideas for improving application design.

Quickly groking and working with deeply nested collections like combinations of dicts and lists (and sometimes other types too) is a common problem, whether it's trying to understand a data set, an API response, or detailed instructions or results from infrastructure as code. Let's talk about current solutions to the problem, their limitations, and how to make that all even easier. Let's talk about making a DeepCollection that can handle anything.

This talk will cover the motivation for a new library, Deep Collections, and some interesting and fun parts of its construction. There are several prior art libraries available, from dpath and Dotty-Dict, to the more recent python-benedict. Motivation for Deep Collections will include a brief discussion of the evolution of the prior art as well as their current limitations. Then we'll discuss how those limitations can be overcome generally, and how to implement some of the more fun and foundational portions of the Deep Collections code to do it.

Software supply chain security is increasingly important to the open source ecosystem, but the learning curve can be steep. Certificate authorities, transparency logs, keys, signing… and even keyless signing! What do these terms all mean and how can a Python developer incorporate tools that make their projects more secure? This talk will provide a high-level overview of the developer-first open source project, Sigstore, within the Python context. We’ll go through each component of Sigstore, including how to sign a software artifact with Cosign, how Fulcio issues certificates, and finally how developers and end users alike can verify claims made on the Rekor public ledger. We’ll discuss how PyPI is leveraging Sigstore to help with verifying and trusting dependencies we all rely on. Finally, we’ll go through a demonstration of creating, publishing, and signing a containerized Python app. The audience will walk away with an understanding of how they can navigate software security more effectively and be better citizens of open source through implementing recommended security practices.

This will be a hands on talk, with an introduction to the Sigstore project, a walk through, and a sample repo for demonstration.

Do you use data visualization to understand data and tell stories? Do you need to visualize big data? Are you interested in leveling up your visualization skills and creating dashboards with HoloViz? This talk is for you!

HoloViz is an open-source high-level Python visualization ecosystem that gives you the superpower to satisfy all your data visualization needs. In this talk, you will learn how to build visualizations easily even for big and multidimensional data, how to turn nearly any notebook into a deployable dashboard, and how to build interactive drill-down exploratory tools for your data and models without having to run a web-technology software development project. You will also learn how to turn your dashboard into WebAssembly and run your dashboard entirely in the browser with the magic of Pyodide and PyScript.

Are you struggling to enforce custom code conventions in large codebases? Learn how to automate this process while maintaining a consistent CI time using Pylint and Pants Build. Attendees will be able to apply the stack in their own codebases right away.

Have you ever been discouraged while trying to enforce code conventions in large codebases? While growing good and long-lasting projects means getting more features, more contributors, and more sub-projects, as a company or community, experience tells one they need rules and code conventions. It gets particularly true with mono-repositories and large frameworks. Enforcing these rules becomes more and more complex and time-consuming as the code base gets larger and the community grows. We will illustrate how to automate such rules enforcement and ease the contributors' life. It turns out Pylint brings the ability to have custom linter rules, and Pants Build helps us manage multi-project repositories, let's merge these two worlds and build an industry-grade project. This talk aims to illustrate the power of this setup and how it can help developers work with large repositories while enforcing specific and strict rules. Takeaways for attendees, that can be put into action right away: * Be able to apply stricter rules to their code while it grows * Keep a constant CI time while scaling, even with more linting rules * Keep tools isolated, portable and safe Base code, config, and extra documentation will be available for attendees to build their setup.

With the vast amount of publicly available data unleashed by the internet's rise, anyone can now document war crimes, track extremist movements, and investigate environmental abuses. This talk will cover how Python tools and developers can accelerate and improve the accessibility of such digital investigations, and hold power to account.

The rise of the internet has driven an explosion in the amount of publicly available data. For those who know where to look, this data can be used to discover things that previously only the intelligence agencies of powerful governments could: today, anyone with an internet connection can document war crimes, track extremist movements, or investigate environmental abuses. These kinds of digital investigations can be both accelerated and made more accessible through the development of open-source software tools. Though the fields of open-source research and open-source software have different origins, they are increasingly converging, and share a common ethos of transparency, reproducibility, and collaboration. This talk will cover some of the ways Bellingcat and other researchers have used Python in their investigations, the intersection of open-source software and open-source research, and how developers can get involved in open-source research.