<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Denizhan Yılmaz on Medium]]></title>
        <description><![CDATA[Stories by Denizhan Yılmaz on Medium]]></description>
        <link>https://medium.com/@denizhanyilmaz9?source=rss-2366bc592631------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*hyqB0HzQzGMfFayc</url>
            <title>Stories by Denizhan Yılmaz on Medium</title>
            <link>https://medium.com/@denizhanyilmaz9?source=rss-2366bc592631------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 05:25:24 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@denizhanyilmaz9/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[ How I Rotated from Analytics Engineer to Software Engineer at Trendyol]]></title>
            <link>https://medium.com/trendyolgroup/how-i-rotated-from-analytics-engineer-to-software-engineer-at-trendyol-2a0fae1a57c6?source=rss-2366bc592631------2</link>
            <guid isPermaLink="false">https://medium.com/p/2a0fae1a57c6</guid>
            <category><![CDATA[sector-rotation]]></category>
            <category><![CDATA[rotation]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[technology]]></category>
            <dc:creator><![CDATA[Denizhan Yılmaz]]></dc:creator>
            <pubDate>Thu, 17 Sep 2026 15:43:28 GMT</pubDate>
            <atom:updated>2026-09-17T15:43:28.803Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Twh2suDHULrDt9QEKMmFJQ.jpeg" /></figure><p>Evolving from a <strong>Data Warehouse Analyst</strong> into a <strong>Full Stack Software Engineer</strong> within the same company wasn’t a linear path — it was a bold pivot driven by passion. Entering Trendyol four years ago with an Industrial Engineering background and shifting into full-stack development on the Data Engineering Growth team might have seemed like a risky restart. But taking a chance on what truly drives you is always worth the leap. Never be afraid to pivot — you’re never starting from scratch, you’re starting from experience.</p><p>Here is the story of how internal opportunities, structured mentorship, and continuous self-learning helped me pivot my technical career toward my long-term goals — all within the same company. If you’re looking to make a similar career leap, here is every step of my rotation process and the lessons I learned along the way.</p><h3>The Spark: From Industrial Engineering to Tech</h3><p>When I graduated from Middle East Technical University (METU) in 2022, my passion was firmly rooted in tech and data. At that time, Machine Learning was the dominant trend — everyone was diving deep into decision trees, KNN algorithms, and feature engineering.</p><p>Driven by this shift, I spent my university years working with public datasets to sharpen my coding, data visualization, and app building skills outside of class. That hands-on curiosity carried into my long-term internship and my final university project, cementing my goal to work in data and software.</p><h3>The DWH &amp; Analytics Engineering Journey</h3><p>Joining Trendyol’s Data Warehouse (DWH) team as a DWH Analyst — and later evolving into an Analytics Engineering scope — was an epic journey. I was given the trust and space to wear multiple hats:</p><ul><li>Designing and modeling data in Enterprise Data Warehouses (EDWs)</li><li>Defining key business metrics and building data pipelines</li><li>Developing BI tools and internal applications</li><li>Working directly with internal stakeholders using a clear product mindset</li></ul><p>As the AI era accelerated, I realized what truly fulfilled me: building end-to-end applications from scratch. Working so closely with data architecture gave me a deep appreciation for complex, large-scale systems. But every time my role required me to build an internal application or tool for our stakeholders, a distinct spark lit up. I found myself obsessing over what was happening under the hood — how the application logic was structured, how the user interface seamlessly communicated with the backend, and how to make the entire system run faster and smoother.</p><p>It was a deeply personal realization: while I loved the power of data, my true passion lay in creation from the ground up. I wanted to master every layer of the software ecosystem — writing robust backends, managing DevOps pipelines, crafting user-centric frontends, and <strong>bringing complete technical ideas to life</strong>. That natural curiosity wasn’t a departure from my roots, but an evolution — I wanted to build the very software systems that power data in the first place.</p><h3>Upskilling, Mentorship, and First Applications</h3><p>That realization sparked a new habit. I dedicated my off-hours to studying software engineering, reading architecture books, and building small projects. But self-study was only part of the puzzle — I needed real-world guidance.</p><p>That’s when I turned to Trendyol’s internal mentorship platform, where team members share their strengths in everything from soft-skills coaching and quick problem-solving to live peer-coding. I connected with Serhat, a senior engineer whose genuine eagerness to teach made all the difference. Coming from an analytical background, transitioning to backend development required a massive mental shift. I initially struggled with software abstraction — high-level concepts like Object-Oriented Design (OOD) principles and Domain-Driven Design (DDD) felt abstract and intimidating compared to data scripting.</p><p>Serhat didn’t just teach me Java syntax; he assigned real-world tasks, conducted detailed code reviews, and patiently walked me through refactoring complex logic into clean, domain-bounded architectures. Over 6 to 8 months, his step-by-step tracking and direct feedback transformed my approach from simply “writing code that works” to “architecting systems that scale.”</p><p>To put my new technical knowledge into practice before making an official leap, I built several internal full-stack software tools for our DWH team:</p><ul><li><a href="https://medium.com/trendyol-tech/smart-scheduling-looker-whitelist-decision-system-455108340d14"><strong>Looker Smart Scheduling:</strong></a> A helper tool for analysts for Looker schedule analysis.</li><li><a href="https://medium.com/trendyol-tech/moving-looks-between-explores-in-looker-using-fastapi-react-1d295522b0a7"><strong>Looker Explore Mover</strong></a><strong>:</strong> A tool to streamline asset migration.</li><li><a href="https://medium.com/trendyol-tech/how-we-made-looker-filters-update-themselves-dynamic-filtering-ce62c26028d0"><strong>Looker Dynamic Filtering</strong></a><strong>:</strong> An app to optimize report filter management.</li><li><strong>Self-Service Scheduling Manager:</strong> An internal tool simplifying pipeline and job management for users.</li></ul><p>These projects proved to me — and my team — that I could bridge the gap between analytics engineering and full-stack software development.</p><h3>The Rotation: Stepping Into Data Engineering</h3><p>As my technical foundation grew, I set up an alignment call with my manager and HRBP. Half-expecting resistance given the scale of the pivot, I was met with total support. Together, we built a clear transition plan, mapping out technical readiness milestones and keeping open lines of communication regarding internal team openings.</p><p>When an internal opening popped up for a <strong>Full Stack Developer</strong> on the <strong>Data Engineering Growth team</strong>, the technical requirements matched my aspirations perfectly. The squad needed someone who could build end-to-end internal platforms to streamline data processes across the organization — a role that called for a mix of software architecture, data domain context, and product thinking.</p><p>With full backing from my leadership, I applied and interviewed. In March 2026, I officially transferred to the Data Engineering team as a Software Engineer.</p><p>“Growth and comfort do not coexist. Change is rarely easy, but stepping into the unfamiliar is where real progress happens.”<em> </em>— Ginni Rometty</p><h3>4 Months In: Lessons &amp; Growth</h3><p>It has been about four months since my rotation, and the learning curve has been incredible. Stepping into a squad heavily focused on big data processing with Spark as the primary Software Engineer was a unique challenge — I had to bridge full-stack practices with heavy data pipelines. In this short time, I have already:</p><ul><li>Contributed to two major project migrations.</li><li>Built one end-to-end internal platform from scratch.</li></ul><p><strong>Taking Ownership Through Migrations:</strong> The two project migrations involved taking full ownership of internal tools that were originally built for our team by other squads. I led the process of re-deploying these projects into our team’s dedicated environment, provisioning new cloud resources, and configuring new domains.</p><p>Bringing these tools in-house eliminated cross-team dependencies, improved infrastructure stability, and gave our squad complete operational autonomy to iterate on features faster without waiting on external roadmaps.</p><p><strong>Building a Unified Digital Marketing Platform</strong>: The application I built from scratch modernized a core operational workflow. What was previously managed through Google Sheets and Google Apps Script is now a unified platform that enables teams to configure and manage digital marketing budgets across multiple channels from a single dashboard.</p><p>This transition eliminated manual spreadsheet errors, drastically reduced operational overhead for marketing operations, and gave leadership real-time control and visibility over multi-channel spend — turning a fragile, manual workflow into a scalable internal product.</p><p>Transitioning into a new engineering squad can feel overwhelming, especially when adjusting to a whole new domain. But Trendyol’s culture made the onboarding seamless. From day one, I was assigned an onboarding buddy who walked me through system architectures and team workflows step by step. Every member of the team has been proactive and eager to help me get up to speed, ensuring that even as the sole software engineer on software-heavy projects, I felt completely backed.</p><h3>Takeaways for Your Own Path</h3><p>This experience reinforced a few core lessons:</p><ul><li><strong>Growth is active, not passive:</strong> Upskilling on your own is key, but leveraging internal resources (like mentorship programs) accelerates your progress twice as fast.</li><li><strong>Build tools that solve real problems:</strong> You don’t need a job title change to start coding. I built small internal applications while still in my DWH role to enhance my capabilities. Be sure to get your hands dirty.</li><li><strong>Don’t wait to feel 100% ready:</strong> Internal rotations exist because companies value domain context and growth mindsets. If you have the drive to learn, the right team will support you through the transition.</li></ul><p>You don’t always need to change companies to change your career trajectory. Sometimes, the right opportunity is just a squad shift away.</p><p>Growth is rarely a straight line, and you shouldn’t have to navigate it alone. If you’re ready to join a culture that empowers your learning curve, supports internal pivots, and helps you map your own path, explore opportunities at<a href="https://careers.trendyol.com/"> Trendyol Careers</a>.</p><p><a href="https://careers.trendyol.com/tr?source=post_page-----455108340d14---------------------------------------">Home - Trendyol Careers</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2a0fae1a57c6" width="1" height="1" alt=""><hr><p><a href="https://medium.com/trendyolgroup/how-i-rotated-from-analytics-engineer-to-software-engineer-at-trendyol-2a0fae1a57c6">🎯 How I Rotated from Analytics Engineer to Software Engineer at Trendyol</a> was originally published in <a href="https://medium.com/trendyolgroup">Trendyol Group</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[ How We Made Looker Filters Update Themselves: Dynamic Filtering]]></title>
            <link>https://medium.com/trendyol-tech/how-we-made-looker-filters-update-themselves-dynamic-filtering-ce62c26028d0?source=rss-2366bc592631------2</link>
            <guid isPermaLink="false">https://medium.com/p/ce62c26028d0</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[looker]]></category>
            <category><![CDATA[data-warehouse]]></category>
            <category><![CDATA[ubuntu]]></category>
            <category><![CDATA[fastapi]]></category>
            <dc:creator><![CDATA[Denizhan Yılmaz]]></dc:creator>
            <pubDate>Thu, 30 Oct 2025 07:11:22 GMT</pubDate>
            <atom:updated>2025-10-30T07:11:22.332Z</atom:updated>
            <content:encoded><![CDATA[<h4>Building Dynamic Filtering at Trendyol to automate report filters and end manual updates.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VHaSvpbpMtA3uBVMAF_IcQ.jpeg" /></figure><h3>🧩 The Problem: When Data Changes Faster Than Your Filters</h3><p>At Trendyol, hundreds of recurring Looker jobs deliver insights daily. But two persistent issues slowed us down:</p><ol><li><strong>Manual input maintenance</strong> — Each time the output is changed in a report, business teams had to manually edit their Look (report) filters.</li><li><strong>Dynamic data sources</strong> — Teams using <strong>Google Sheets</strong> for campaign or product data couldn’t sync updates automatically for another report.</li></ol><p>Whenever a report’s <strong>output results</strong> or <strong>filter values</strong> change, users have to manually update their Looker filters to keep reports accurate.</p><h3>🧾 Real-World Examples</h3><h4>Looker Reports Feeding Looker Reports</h4><p>Imagine this:<br>Yesterday, <em>Computers</em> and <em>Jackets</em> were the top-selling product categories.<br>Today, <em>Shoes</em> and <em>Home Accessories</em> take the lead.</p><p>The report analyzing “Top Selling Products” needs to update its filters accordingly. But until now, report owners had to do this <strong>manually every single day</strong> — opening the Look, editing filters, saving, and re-running schedules.</p><h4>External Sheets Feeding Looker</h4><p>Now consider another scenario:<br>A supplier company sends hourly sales updates to Trendyol analysts through a <strong>Google Sheet</strong>. The sheet includes product barcodes, refreshed automatically every hour.</p><p>However, the Looker reports that use those barcodes as filters don’t update automatically.<br>So analysts had to <strong>manually copy the new barcode values</strong> from the Google Sheet into the Looker filters every hour just to keep reports accurate.</p><p>Here is the question to be asked:</p><blockquote><em>“Can Looker reports and its filters update themselves when upstream outputs change?”</em></blockquote><p>That question led to <strong>Dynamic Filtering</strong>.</p><h3>⚙️ The Solution: <strong>Dynamic Filtering</strong></h3><p><strong>Dynamic Filtering</strong> automates Looker schedule updates based on dynamic inputs from <strong>Google Sheets</strong> or <strong>other Looks</strong>, using a FastAPI backend and a native Looker extension.</p><p>It supports:</p><ul><li><strong>Looker</strong> → <strong>Looker</strong> and <strong>GSheet </strong>→ <strong>Looker</strong> flows</li><li><strong>APScheduler</strong>-based automation</li><li><strong>BigQuery</strong> job persistence</li><li>Editable cron-style triggers through Looker UI</li></ul><h3>🧠 Architecture Overview</h3><p>High-Level Design</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yjQE66OrM6xFh6gNyNdebQ.png" /></figure><h3>How The Data Really Flows?</h3><p>If the source is <strong>Google Sheet</strong>:</p><ol><li>User updates Google Sheet.</li><li><strong>APScheduler</strong> (in FastAPI) triggers on schedule → <strong>Backend</strong> reads <strong>BigQuery</strong> for active jobs.</li><li><strong>Backend</strong> reads the <strong>GSheet</strong> (headers/rows) → assembles filter payloads.</li><li><strong>Backend</strong> calls <strong>Looker SDK</strong> → applies filters to the Look.</li><li><strong>Backend</strong> logs results back to <strong>BigQuery</strong>; <strong>Extension</strong> reads status from the backend.</li></ol><p>If the source is <strong>Looker</strong>:</p><ol><li><strong>APScheduler</strong> (in FastAPI) triggers on schedule → <strong>Backend</strong> reads <strong>BigQuery</strong> for active jobs.</li><li><strong>Backend</strong> runs the <strong>Look</strong> → assembles filter payloads.</li><li><strong>Backend</strong> calls <strong>Looker SDK</strong> → applies filters to the Look.</li><li><strong>Backend</strong> logs results back to <strong>BigQuery</strong>; <strong>Extension</strong> reads status from the backend.</li></ol><h3><strong>🧩 BigQuery Table</strong></h3><pre>CREATE TABLE dataset.input_controller_scheduled_jobs (<br>  schedule_id        STRING,<br>  source_type        STRING,<br>  from_look_id       INT64,<br>  to_look_id         INT64,<br>  gsheet_doc_id      STRING,<br>  gsheet_column_name STRING,<br>  to_filter_field    STRING,<br>  from_filter_field  STRING,<br>  minute             STRING,<br>  hour               STRING,<br>  day                STRING,<br>  month              STRING,<br>  day_of_week        STRING,<br>  email              STRING,<br>  created_at         TIMESTAMP,<br>  updated_at         TIMESTAMP,<br>  is_active          BOOL<br>);</pre><h3>🛠️ Backend Deployment on Ubuntu Server</h3><h4>1. Install Requirements</h4><pre>sudo apt update<br>sudo apt install -y python3-pip python3-venv nginx<br>python3 -m venv /opt/input-controller-venv<br>source /opt/input-controller-venv/bin/activate<br>pip install fastapi uvicorn apscheduler looker-sdk google-cloud-bigquery gspread</pre><h4>2. Running Scheduled Jobs</h4><p>The summary of the backend service is as follows:</p><pre>def run_scheduled_job(schedule: dict):<br>    # 1️⃣ Fetch source data<br>    if schedule[&quot;source_type&quot;] == &quot;gsheet&quot;:<br>        sheet = gclient.open_by_key(schedule[&quot;gsheet_doc_id&quot;]).sheet1<br>        rows = sheet.get_all_records()<br>        from_output = [str(r.get(schedule[&quot;gsheet_column_name&quot;])) for r in rows]<br>   ...<br><br>    # 2️⃣ Update target Look filter<br>    to_look = sdk.look(schedule[&quot;to_look_id&quot;])<br>    ...<br>    to_query.filters[schedule[&quot;to_filter_field&quot;]] = &quot;,&quot;.join(from_output[:10000])<br><br>    # 3️⃣ Create new query and update Look<br>    new_query = sdk.create_query(models40.WriteQuery(...))<br>    sdk.update_look(look_id=schedule[&quot;to_look_id&quot;],<br>                    body=models40.WriteLookWithQuery(query_id=new_query.id))<br><br>    # 4️⃣ Update all associated scheduled plans<br>    update_schedules_for_look(<br>        look_id=schedule[&quot;to_look_id&quot;],<br>        field_name=schedule[&quot;to_filter_field&quot;],<br>        new_values=from_output,<br>    )</pre><h4>3. FastAPI Endpoints</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/632/1*Nhu_UHznRSpPbWTwtn39BA.png" /></figure><h3>🛠️ Frontend Deployment with React on Looker</h3><p>On the frontend side of this project, we used React with the capabilities of Looker to deploy with <em>bundle.js</em> on a project in Looker. Here is the main page of the Dynamic Filtering app on Looker:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*izZ4aQkLBm17FeUnu7_8uQ.png" /></figure><p>Here when the form is submitted, the trigger part is opened and the user can automate the operation on a schedule. When the trigger is submitted, it is logged on BigQuery table.</p><p>On <strong>Your Triggers </strong>tab, user can see and monitor its own triggers as below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/867/1*Eo6J__aomWNKHPM3icaI8A.png" /></figure><p>On this tab, user can edit the trigger it scheduled, and start and stop the automation for each trigger they created.</p><h3>📈 Results</h3><p><strong>Pros of Dynamic Filtering</strong>:</p><ul><li><strong>Near-zero manual upkeep:</strong> Routine filter edits and report tweaks have been eliminated.</li><li><strong>Self-serve control:</strong> Analysts can <strong>Edit </strong>the triggers they created, <strong>Start or Stop</strong> runs when it is no longer needed or can be stopped for some time they need.</li><li><strong>Mass updates at scale:</strong> Apply <strong>10,000+</strong> filter values in a single run.</li></ul><h3>🧭 Takeaways</h3><p>The <strong>Looker Dynamic Filtering</strong> bridges operational BI and automation — <br>replacing manual filter updates with a self-maintaining orchestration layer.</p><p>By combining <strong>Looker SDK</strong>, <strong>BigQuery</strong>, <strong>APScheduler</strong>, and <strong>GSheet APIs</strong>, we made our analytics pipeline both adaptive and scalable.</p><blockquote>Small system, big leverage — the kind of automation that makes BI invisible.</blockquote><p><strong>About Us</strong></p><p>Ready to take your career to the next level? Join our dynamic team and make a difference at Trendyol.</p><p><a href="https://careers.trendyol.com/tr?source=post_page-----455108340d14---------------------------------------">Home - Trendyol Careers</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ce62c26028d0" width="1" height="1" alt=""><hr><p><a href="https://medium.com/trendyol-tech/how-we-made-looker-filters-update-themselves-dynamic-filtering-ce62c26028d0">🚀 How We Made Looker Filters Update Themselves: Dynamic Filtering</a> was originally published in <a href="https://medium.com/trendyol-tech">Trendyol Tech</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Moving Looks Between Explores in Looker Using FastAPI & React]]></title>
            <link>https://medium.com/trendyol-tech/moving-looks-between-explores-in-looker-using-fastapi-react-1d295522b0a7?source=rss-2366bc592631------2</link>
            <guid isPermaLink="false">https://medium.com/p/1d295522b0a7</guid>
            <category><![CDATA[backend-development]]></category>
            <category><![CDATA[looker]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[fastapi]]></category>
            <category><![CDATA[react]]></category>
            <dc:creator><![CDATA[Denizhan Yılmaz]]></dc:creator>
            <pubDate>Fri, 27 Jun 2025 13:28:46 GMT</pubDate>
            <atom:updated>2025-06-27T13:46:50.262Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7V-93O3JgEO7HM4Bbu1LTw.jpeg" /><figcaption>AI Generated Image</figcaption></figure><p>As part of our DWH (Data Warehouse) environment at Trendyol, we maintain two key Looker Explore resources to support varying analytical needs:</p><p>At Trendyol, we utilize two main Looker resources:</p><ul><li>Our <strong>T-1 data mart</strong> holds rich, detailed data,</li><li><strong>Near Real Time</strong> resource is designed for quicker, less detailed, and more operational insights.</li></ul><p>Both are made accessible to business users through Looker explores, enabling self-service BI. Although these Looker explores have similar data structures, they actually pull from different underlying data sources and have distinct definitions.</p><p>A common scenario is that users start by creating a report in the <strong>near-real-time explore</strong> to get quick insights. Later, when they want to <strong>monitor it regularly</strong> or <strong>combine it with richer dimensions</strong>, they decide to move the report to the <strong>T-1 explore</strong>. However, this transition isn’t simple. That’s because the same report (or ‘Look’) cannot be directly reused due to inconsistencies in field or view names. This forces users to manually recreate the entire report, a process that can take <strong>over 10 minutes</strong>.</p><p>With this tool, we’ve reduced that time to <strong>~30 seconds</strong> by automating the field and filter mapping between explores — saving valuable time and cognitive load for analysts.</p><h3>🍪 The Problem</h3><p>For analysts, creating a report with identical fields and filters on a Looker Explore that has different names for its views and fields is a challenging process. Some reports may have over 30 fields and re-creating it from scratch in a new explore takes so much of their time.</p><h3>🚀 The Solution: “Move Smarter, Not Harder”</h3><p>This tool solves that by:</p><ol><li>Fetching the Look’s original fields and filters.</li><li>Mapping them using a Google Sheet that defines field-to-field translations.</li><li>Creating a new Look in the desired Explore with translated fields.</li><li>Logging everything to BigQuery.</li></ol><h3>🦢 Bottlenecks</h3><p>When you want to duplicate an existing report (a ‘Look’) from one Explore (Explore A) to another (Explore B) using the Looker SDK, even if they have similar data structures (though field names might differ slightly), you’ll encounter these obstacles:</p><ul><li>Field names do not exactly match</li><li>View names do not match</li><li>Filters may break</li><li>Tedious manual re-creation of reports</li></ul><p>For instance, <em>sales_pbl_trendyol_order_trx.total_price</em> in <strong>T-1</strong> Looker Explore might be named <em>sales_nrt_pbl_order_trx.total_price</em> in <strong>Near Real Time</strong> Looker Explore. Another example is, <em>store_status </em>can be named<em> current_store_status </em>in the other one, which needs manual mapping effort.</p><h3>📄 How It Works</h3><h3>The Flow</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MK3DREWqKZ2mtbKUUQXdRQ.png" /><figcaption>Flow Diagram of the Explore Mover Application</figcaption></figure><h3>Frontend (React)</h3><p>The interface is clean and minimal:</p><ul><li>Look ID input</li><li>Folder ID selector</li><li>Look Title input</li><li>Directions: <em>Sales (Near Real Time) to Sales</em> and <em>Sales to Sales (Near Real Time)</em></li><li>User e-mail (for logging)</li></ul><p>Once you hit “Map &amp; Create Look”, a call is made to the FastAPI backend.</p><p><strong>Sample React snippet:</strong></p><pre>try {<br>  const res = await fetch(`https://xxx.xxx.xxx.xxx:port/map_fields?look_id=${lookId}&amp;direction=${direction}&amp;look_title=${encodeURIComponent(lookTitle)}&amp;folder_id=${folderId}&amp;user_email=${userEmail}`)<br>  if (!res.ok) throw new Error(await res.text())<br>  const data = await res.json()<br>  setResponse(data)<br>} catch (err) {<br>  setError(err.message)<br>} finally {<br>  setLoading(false)<br>}</pre><h3>Backend (FastAPI)</h3><p>The backend performs the following:</p><ul><li>Uses Looker SDK to fetch the original Look query.</li><li>Applies field &amp; filter mappings from a Google Sheet:</li><li>Worksheet: nrt_to_tmone_fields</li><li>Worksheet: tmone_to_nrt_fields</li><li>Creates a new Look with translated values.</li><li>Logs the request into a BigQuery table<em> (explore_mover_logs)</em> for auditability.</li></ul><p><strong>Gsheet Field Mapping Sample:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gA_ojxMN_rYIWn3CDwGR5g.png" /><figcaption>Field Mapping on Google Sheet</figcaption></figure><p><strong>Sample backend code for field mapping:</strong></p><pre>…<br>n_to_one_map = {str(k).strip(): str(v).strip() for k, v in zip(n_to_one[&#39;nrt_field&#39;], n_to_one[&#39;tmone_field&#39;])}<br>one_to_n_map = {str(k).strip(): str(v).strip() for k, v in zip(one_to_n[&#39;tmone_field&#39;], one_to_n[&#39;nrt_field&#39;])}<br>…</pre><p><strong>Sample backend code for endpoint creation:</strong></p><pre>@app.get(&quot;/map_fields&quot;)<br>def map_fields(direction: str = &quot;nrt_to_tmone&quot;, look_id: int = 85759, look_title: str = &quot;Mapped Look&quot;, folder_id: str = &quot;&quot;, user_email: str = &quot;&quot;):<br>    if not folder_id:<br>        raise HTTPException(status_code=400, detail=&quot;❌ &#39;folder_id&#39; is required&quot;)<br><br>    try:<br>        q = sdk.look(look_id=str(look_id)).query<br><br>        if direction == &quot;nrt_to_tmone&quot;:<br>            mapped_fields = [n_to_one_map.get(f, f) for f in q.fields]<br>            mapped_filters = {n_to_one_map.get(k, k): v for k, v in q.filters.items()}<br>            model = sales_bq&quot;<br>            view = &quot;sales_pbl_trendyol_order_trx&quot;<br>        elif direction == &quot;tmone_to_nrt&quot;:<br>            mapped_fields = [one_to_n_map.get(f, f) for f in q.fields]<br>            mapped_filters = {one_to_n_map.get(k, k): v for k, v in q.filters.items()}<br>            model = sales_nrt_bq&quot;<br>            view = &quot;sales_nrt_pbl_order_trx&quot;<br>        else:<br>            raise HTTPException(status_code=400, detail=&quot;❌ Invalid direction&quot;)</pre><h3>Logging Example</h3><p>Each action is logged as a single row:</p><pre>{<br>  &quot;status&quot;: &quot;✅ success&quot;,<br>  &quot;mapped_fields&quot;: […],<br>  &quot;mapped_filters&quot;: {…},<br>  &quot;from_look_id&quot;: 85759,<br>  &quot;new_look_id&quot;: 123456,<br>  &quot;new_query_id&quot;: 78910,<br>  &quot;folder_id&quot;: &quot;1&quot;,<br>  &quot;user_email&quot;: &quot;denizhan.yilmaz@xmail.com&quot;,<br>  &quot;direction&quot;: &quot;tmone_to_nrt&quot;,<br>  &quot;timestamp&quot;: &quot;2025–05–30T12:00:00&quot;<br>}</pre><h3>🚧 Technologies Used</h3><ul><li><strong>FastAPI</strong>: for the API backend</li><li><strong>Looker SDK v4.0</strong>: to interact with Looker</li><li><strong>React</strong>: for the user interface</li><li><strong>Google Sheets API</strong>: to manage field mappings</li><li><strong>BigQuery</strong>: for logging usage history</li></ul><h3>📊 Benefits</h3><ul><li>Saves hours of analyst and planning teams’ work</li><li>Promotes consistency between Looker environments</li><li>Audit trail available in BQ</li><li>Flexible enough to plug in other explore mappings later</li></ul><h3>🔧 Setup and Deployment Notes</h3><ul><li>Use code below in the server to run the application</li></ul><pre>uvicorn main_fastapi:app - host 0.0.0.0 - port 8000 to run</pre><ul><li>If deploying on a server, consider setting up SSL or serving behind NGINX</li><li>Your <em>credentials.json</em> should be authorized for both Google Sheets and Drive access</li><li>Add CORS middleware for local React usage</li></ul><h3>🍕 The Final Product</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*awxU_0CIgGzwIrJLmLlMcw.png" /></figure><h4>💡 Reusability Beyond NRT vs. T-1</h4><p>While the most common use case is shifting between <strong>near-real-time and T-1 explores</strong>, the tool is also useful in:</p><ul><li>Switching between <strong>aggregate vs. detailed explores</strong> that offer the same metrics with different levels of granularity</li><li>Cases where explore names or field naming conventions diverge between environments or business domains</li><li>Any situation where the <strong>underlying data logic remains the same</strong>, but the Looker model and view differs</li></ul><p>For routine transitions between consistent datasets presented through different explores, it’s a time-saver and a productivity boost.</p><h3>📊 Final Thoughts</h3><p>To ensure better performance and optimal resource usage, we recommend using <strong>T-1 explores</strong> over <strong>Near Real Time explores</strong> in Looker. While real-time explores are convenient and often favored by users, their popularity can lead to system overload and performance bottlenecks. Not only for this type of relationship, with the proper field mapping, we wanted to demonstrate this method can be used for different types of Looker Looks — Reports.</p><p>As a result, analysts and planning teams frequently migrate their reports from <strong>real-time explores</strong> to more stable <strong>T-1 explores</strong>. However, manually recreating reports — especially those with dozens of fields and filters — can be both <strong>painful and time-consuming</strong>.</p><p>To address this, we built a <strong>custom Looker Extension</strong> from scratch. It automates the migration process, allowing users to <strong>recreate a Look with a new explore in just ~30 seconds</strong> — preserving maximum number of fields and filters with minimal effort.</p><h3>About Us</h3><p>Ready to take your career to the next level? Join our dynamic team and make a difference at Trendyol.</p><p><a href="https://careers.trendyol.com/tr?source=post_page-----455108340d14---------------------------------------">Home - Trendyol Careers</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1d295522b0a7" width="1" height="1" alt=""><hr><p><a href="https://medium.com/trendyol-tech/moving-looks-between-explores-in-looker-using-fastapi-react-1d295522b0a7">Moving Looks Between Explores in Looker Using FastAPI &amp; React</a> was originally published in <a href="https://medium.com/trendyol-tech">Trendyol Tech</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[“Smart Scheduling: Looker Whitelist Decision System”]]></title>
            <link>https://medium.com/trendyol-tech/smart-scheduling-looker-whitelist-decision-system-455108340d14?source=rss-2366bc592631------2</link>
            <guid isPermaLink="false">https://medium.com/p/455108340d14</guid>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[looker]]></category>
            <category><![CDATA[automation]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Denizhan Yılmaz]]></dc:creator>
            <pubDate>Mon, 22 Jul 2024 08:52:19 GMT</pubDate>
            <atom:updated>2024-07-22T08:52:55.585Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>Smart Scheduling: </strong>Looker Whitelist Decision System</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/499/1*Sn0WeVKxmzarAHzQc0Vl8Q.jpeg" /></figure><p>In this article I will mention how we implemented a decision support system to a manual process we conduct for responding business teams’ Looker Schedules Whitelist Requests. First, I want to mention what this Whitelist process is and how much effort we spend every week.</p><h3>Overview</h3><p>We work with business teams and individuals to schedule Looker reports for operational analyses and reporting. These reports, exported as CSV, Excel, or image files, are sent via email, Google Sheets, etc., on a recurring basis (daily, weekly, hourly).</p><p>Schedules may be set for specific periods, such as during events or for daily order checks over months. Once these periods end, the reports can become redundant and burden the server. Additionally, most reports are requested around 9 am, causing server overload due to limited slot capacity. Thus, we channel report owners to schedule their report for another time, like 9.05 am to prevent overload. To address this we, DWH Analysts, implemented a whitelist procedure. If schedules are not approved by an analyst and whitelisted, they are automatically deleted after one month.</p><p>To maintain database performance and prevent unnecessary reports from bogging down Trendyol’s system, on-call analysts review all schedules requests weekly, adhering to specific rules as Looker Explore, filters used in the report and cron-time-based rules etc.. Approximately 200 to 250 scheduling requests are received each week, and analyzing each report takes about 1 to 2 minutes. This review process is a significant, time-consuming responsibility for the on-call analyst.</p><p>For this project, our goal was to automate the process using a Machine Learning application, thereby reducing the time and effort involved. Initially, we considered creating a checklist for business owners, allowing them to perform the checks we usually conduct before the whitelist process. However, this approach could lead to unpredictable number schedule additions without a supervised check. Therefore, we opted to develop an application that assists in determining whether a report should be added to the whitelist. This solution is more reliable and sustainable for an operation of this scale.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ONEfDG7Rm6OtMzdSHsae-Q.png" /><figcaption>Whitelist Scheduling Process</figcaption></figure><p>Our main aim has been to maximize customer-business satisfaction while minimizing database slot use with minimum human interaction.</p><h3>Project Steps</h3><ol><li>Collect Data</li><li>Parse and Clean the Data</li><li>Implement and Analyze Machine Learning Algorithms</li><li>Deploy</li><li>Gather Feedback and Improve Results</li></ol><h4>Collect Data</h4><p>We collect whitelist requests in a Google Drive sheet with its Schedule Name, ID and Creation Date. Using the ID’s, we can observe the schedule run date and time in cron format, Explore used in Looks, fields and filters applied on the Look etc. details via Looker sources. To collect data in our application, we implemented <a href="https://cloud.google.com/looker/docs/api-sdk">Looker SDK</a> giving the <em>scheduleId </em>as input and Look details as output:</p><pre>import looker_sdk<br>from looker_sdk import models40<br><br>sdk = looker_sdk.init40(&quot;looker.ini&quot;)<br><br>### GET QUERY ID FROM LOOK<br>query = sdk.look(look_id=&#39;60971&#39;).query<br><br>…<br><br>### CREATE NEW QUERY AND ADD A FILTER<br>new_query = sdk.create_query(body = models40.WriteQuery(model=&quot;system__activity&quot;,<br>view = query.view,<br>fields = query.fields,<br>filters = {&quot;scheduled_plan.id&quot;:str(scheduleId)}))<br><br>#### UPDATE LOOK WITH NEW QUERY ID<br>sdk.update_look(look_id=&quot;60971&quot;,<br>body = models40.WriteLookWithQuery(query_id = new_query.id))<br><br>#### RUN LOOK<br>result = sdk.run_look(look_id=&quot;60971&quot;, result_format=&quot;json&quot;)<br><br>…</pre><p>Thanks to Looker SDK, now the collected data is a json file that includes all the necessary fields analysts use to determine whether the schedule needs to be approved or not.</p><h4>Parse and Clean the Data</h4><p>The collected data includes report filters in JSON format. We need to verify necessary filters, such as whether the Business Unit Name is filtered. The data source used for the report is crucial, as scheduling hourly reports from a daily source is inefficient. Additionally, most reports have date filters like “Order Date: 2023/01/01 to 2024/01/01” or “last 4 weeks.” A function must accurately parse these periods and use as a criteria to approve the request or not to prevent server lags and timeout errors.</p><p>After parsing and collecting filters applied, we filled or deleted null rows with null values to make them suitable to use as input in the machine learning algorithm.</p><h4>Implement and Analyze Machine Learning Algorithms</h4><p>The requests are either approved or not, so we need to retrieve binary results from the algorithm. The fields used in the model are categorized as either categorical or numerical variables. For example, <em>Time Period in Days</em> is a numerical variable, while <em>cron_schedule</em> is a categorical variable.</p><p>We have around 7,000 requests in our dataset, with a 10% rejection rate, making it an imbalanced dataset. Initially, we applied data balancing techniques such as undersampling and oversampling to Random Forest Classification and Gradient Boosting algorithms to create our model. Better results were received with SMOTE with Gradient Boosting algorithm. SMOTE stands for <em>Synthetic Minority Oversampling Technique</em>. This technique is used to address imbalanced datasets and create synthetic nodes for the minority class in the learning dataset.</p><pre>from sklearn.ensemble import GradientBoostingClassifier<br>from sklearn.metrics import confusion_matrix, classification_report, accuracy_score<br>import pandas as pd<br>import category_encoders as ce<br>from sklearn.model_selection import train_test_split<br><br>…<br><br># Load the parsed training data<br>df = pd.read_csv(&#39;./trains/train_df.csv&#39;, index_col=0)<br><br># Ordinal encoding for categorical variables<br>encoder = ce.OrdinalEncoder(cols=[…])<br><br>…<br><br># Split the data into training and test sets<br>X = df.drop(&#39;Is Added to Whitelist?&#39;, axis=1)<br>y = df[&#39;Is Added to Whitelist?&#39;]<br>X_train, X_test, y_train, y_test = train_test_split(X, y, train_size=0.67, random_state=1)<br><br># Apply ordinal encoding to the training and test sets<br>X_train = encoder.fit_transform(X_train)<br>X_test = encoder.transform(X_test)<br><br># Initialize and train the Gradient Boosting Classifier<br>gbc = GradientBoostingClassifier(n_estimators=300, random_state=100)<br>gbc.fit(X_train, y_train)<br><br># Predict the test set results<br>y_pred = gbc.predict(X_test)<br><br># Evaluate the model<br>print(f&#39;Model accuracy score: {accuracy_score(y_test, y_pred):.4f}&#39;)<br>print(&#39;Feature importances:&#39;)<br>print(pd.Series(gbc.feature_importances_, index=X_train.columns).sort_values(ascending=False))<br>print(&#39;Confusion Matrix:&#39;)<br>print(confusion_matrix(y_test, y_pred))<br>print(&#39;Classification Report:&#39;)<br>print(classification_report(y_test, y_pred))</pre><p>Oversampling responded to our dataset well with the initial results below for Random Forest and XGBoost algorithms.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/844/1*m8Z01TbIyXH47dE7mklNtw.png" /><figcaption>Initial Results for Random Forest and XGBoost Algorithms</figcaption></figure><p>These were the initial results. Trying different variations of inputs and sampling methods we now use the Gradient Boosting algorithm to support our decisions at the first base because it resulted in flagging “No” decisions more accurately, which means false positive rates were more promising with XGBoost. With continuous improvement and replies we aim to fully automate this process with minimum human interaction. The model creation is as below:</p><h4>Deploy</h4><p>In the first phase we adapted our Google Sheet where we collect requests, we added results from our model and compared algorithm results with our decisions and analyzed edge cases and common model mistakes to enhance our model’s performance.</p><h3>Conclusion</h3><p>We are in the Gather Feedback and Improve Results step of the project. This project needs a continuous learning mechanism as standards, states, and decision criteria can change over time. Implementing a decision support system for Looker Schedules Whitelist Requests has streamlined our process. Using the Looker SDK for data collection, filter parsing, and machine learning, we’ve reduced manual effort and improved accuracy.</p><p>Gradient Boosting outperformed Random Forest, achieving high accuracy. During deployment, we refined our model and integrated it with Google Sheets for real-time feedback. This system maximizes efficiency and satisfaction while minimizing database load, aiming for full automation with minimal human intervention.</p><p>By automating the decision-making process, we’ve significantly reduced the need for manual reviews and checks at the first step, where we use it as a decision support system. Previously, analysts spent considerable time manually verifying and processing requests, but soon the machine learning model will handle these tasks efficiently, ultimately eliminating the need for human intervention. This transition will lead to more consistent and objective decision-making, allowing our team to focus on more strategic activities. As a result, the on-call analyst will spend 3 hours less on these tasks.</p><h3>About Us</h3><p>Ready to take your career to the next level? Join our dynamic team and make a difference at Trendyol.</p><p><a href="https://careers.trendyol.com/">Home - Trendyol Careers</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=455108340d14" width="1" height="1" alt=""><hr><p><a href="https://medium.com/trendyol-tech/smart-scheduling-looker-whitelist-decision-system-455108340d14">“Smart Scheduling: Looker Whitelist Decision System”</a> was originally published in <a href="https://medium.com/trendyol-tech">Trendyol Tech</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>