forked from bfijalkowski/KWT-2024
48 KiB
48 KiB
Komputerowe wspomaganie tłumaczenia
9,10. Web scraping [laboratoria]
Rafał Jaworski (2021)
Jak dobrze wiemy, w procesie wspomagania tłumaczenia oraz w zagadnieniach przetwarzania języka naturalnego ogromną rolę pełnią zasoby lingwistyczne. Należą do nich korpusy równoległe (pamięci tłumaczeń), korpusy jednojęzyczne oraz słowniki. Bywa, że zasoby te nie są dostępne dla języka, nad którym chcemy pracować.
W tej sytuacji jest jeszcze dla nas ratunek - możemy skorzystać z zasobów dostępnych publicznie w Internecie. Na dzisiejszych zajęciach przećwiczymy techniki pobierania tekstu ze stron internetowych.
Poniższy kod służy do ściągnięcia zawartości strony (w formacie HTML do zmiennej) oraz do wyszukania na tej stronie konkretnych elementów. Przed jego uruchomieniem należy zainstalować moduł BeautifulSoup:
pip3 install beautifulsoup4
import warnings
warnings.filterwarnings('ignore')import requests
from bs4 import BeautifulSoup
url='https://epoznan.pl'
page = requests.get(url)
soup = BeautifulSoup(page.content, 'html.parser')
headers = soup.find_all('h3', {'class':'postItem__title'})
print('\n'.join([header.get_text() for header in headers]))Wypadek na Dąbrowskiego. Jedno z aut dachowało Znów gęsty dym nad miastem IMGW ostrzega. W 19 powiatach naszego regionu! Kolejna inwestycja zakończona. Na samym Starym Rynku Lech przegrał z Legią w Poznaniu. Dwa gole samobójcze W każdym elektrycznym Volkswagenie ID znajdziemy "kawałek Poznania" Takie parkomaty stawiają na poznańskich ulicach. Będzie dużo nowości. Za parkowanie zapłacimy kartą i blikiem Policjanci publikują wizerunek i szukają jednego z klientów poznańskiego pubu Zielone naklejki na lampach w poznańskich parkach. Co oznaczają? Nowy sposób oszustów. Mogą pisać takie wiadomości Szukamy Ciebie! Najlepsze oferty pracy dla osób z niepełnosprawnościami Park przemysłowy pod Poznaniem sprzedany za 92,5 mln euro Warta zagrała w Krakowie "Biały dym" nad miastem. Po przejeździe pociągu z kibicami Legii Warszawa? Czytelnik zwraca uwagę na kolejne niebezpieczne skrzyżowanie w Poznaniu. "W tym miejscu nie ma znaku" Jeżdżą i robią ważne badania w Wielkopolsce. W maju zaplanowanych jest jeszcze 135 postojów Pierożki Dim Sum z Para Bar Rataje ze specjalną zniżką! Ogromna strata zakochanej kobiety. Marynarz już miał wysłać do niej walizkę Dziś mecz podwyższonego ryzyka. Czekają nas spore utrudnienia w okolicach stadionu! Miejsce z Poznania może zostać Cudem Polski 2024. Trwa głosowanie Kolejna inwestycja na Starołęce. Ruszyła budowa Część poznaniaków nie musi dziś płacić za bilet! Dla kogo darmowa komunikacja? Nie wszyscy mieszkańcy poznańskich osiedli mogą iść dziś na wybory Portowo - tu życie płynie Zaginął 42-latek. "Przed północą oddalił się w nieznanym kierunku" Ostatniej nocy na niebie znów rozegrał się niezwykły spektakl Policjanci z Wildy publikują wizerunek i szukają tego mężczyzny Czterdzieści kotów w jednym domu. Nie wszystkie żyły ZDM nie łata dziur w jezdniach? "Od 2 miesięcy zgłaszam i zero reakcji" Kino plenerowe dziś na Łazarzu. "Za free!"
Ćwiczenie 1: Napisz funkcję do pobierania nazw towarów z serwisu Ceneo.pl. Typ towaru, np. telewizor, pralka, laptop jest parametrem funkcji. Wystarczy pobierać dane z pierwszej strony wyników wyszukiwania.
def get_names(article_type):
url = f"https://m.ceneo.pl/;szukaj-{article_type}"
page = requests.get(url)
soup = BeautifulSoup(page.content, 'html.parser')
prod_names = soup.find_all('div', {'class':'list-prod-name'})
prod_names = [prod_name.get_text().strip() for prod_name in prod_names]
return prod_namesget_names('telewizor')[-3:]['Telewizor LED TCL 43P638 43 cale 4K UHD', 'Telewizor LED LG 32LQ630B6LA 32 cale HD Ready', 'Telewizor LED Xiaomi Mi LED TV P1 43 cale 4K UHD']
get_names('pralka')[-3:]['Pralka Electrolux PerfectCare 600 EW6SM404WP', 'Pralka MPM MPM-4610-PH-03', 'Pralka Whirlpool FFS 7259 B EE']
get_names('laptop')[-3:]['Laptop Acer Nitro 16 AN16-41 R7-7840HS/16GB/1TB/RTX4070/W11 165Hz', 'Laptop Lenovo IdeaPad Slim 3 15ABR8 15,6"/Ryzen5/16GB/512GB/NoOS (82XM009NPB)', 'Laptop Dell Latitude 7640 16"/i7/32GB/1TB/Win11 (N010L764016EMEA_VP)']
W ten sposób pobieramy dane z jednej strony. Nic jednak nie stoi nam na przeszkodzie, aby zasymulować przełączanie stron.
Ćwiczenie 2: Zaobserwuj, jak zmienia się url strony podczas przechodzenia do kolejnych stron wyników wyszukiwania na Ceneo.pl. Wykorzystaj tę informację i uruchom funkcję get_names() na więcej niż jednej stronie wyników.
def scrape_names(article_type: str, num_pages: int):
url = f"https://m.ceneo.pl/;szukaj-{article_type}"
page = requests.get(url)
prod_names = []
for page_idx in range(1, num_pages+1):
tmp_page = requests.get(f"{page.url};0020-30-0-0-{page_idx}.htm")
soup = BeautifulSoup(tmp_page.content, 'html.parser')
tmp_prod_names = soup.find_all('div', {'class':'list-prod-name'})
tmp_prod_names = [prod_name.find('span') for prod_name in tmp_prod_names]
tmp_prod_names = [prod_name.get_text().strip() for prod_name in tmp_prod_names]
prod_names.extend(tmp_prod_names)
return prod_namesscrape_names('telewizor', 3)[-3:]['Telewizor LED LG 70UR80003LJ 70 cali 4K UHD', 'Telewizor QLED Samsung QE85Q70C 85 cali 4K UHD', 'Telewizor LED Hisense 32A4K 32 cale 4K UHD']
scrape_names('pralka', 3)[-3:]['Pralka Bosch Serie 4 WGG0440EPL', 'Pralka Haier I-Pro 3 HW90-BP14939', 'Pralka Samsung WW11BGA046AE']
scrape_names('laptop', 3)[-3:]['Laptop Dell Latitude 3340 13,3"/i3/8GB/256GB/Win11 (L1333400023853SA)', 'Laptop HP ZBook Firefly 14 G8 14"/i7/16GB/512GB/Win10 (2C9R0EA)', 'Laptop HP 17-CN3053 17,3"/i5/32GB/2TB/Win11 (7F8G5UA)']
Technika pobierania treści z Internetu jest szczególnie efektywnym sposobem na pozyskiwanie dużych ilości tekstu. Poniższy fragment kodu służy do ściągnięcia całości tekstu ze strony.
import re
url = "https://www.yahoo.com"
page = requests.get(url)
soup = BeautifulSoup(page.content, 'html.parser')
# usunięcie elementów script i style
for script in soup(["script", "style"]):
script.extract() # usuń element
# pobierz tekst
text = soup.get_text()
# usuń wielokrotne białe znaki
text = re.sub(r"\s+", " ", text)
print(text)Yahoo | Mail, Weather, Search, Politics, News, Finance, Sports & Videos News Today's news US Politics World Tech Reviews and deals Audio Computing Gaming Health Home Phones Science TVs Climate change Health Science 2024 election Originals The 360 Life Health COVID-19 Fall allergies Health news Mental health Relax Sexual health Studies The Unwind Parenting Family health So mini ways Style and beauty It Figures Unapologetically Horoscopes Shopping Buying guides Food Travel Autos Gift ideas Buying guides Entertainment Celebrity TV Movies Music How to Watch Interviews Videos Shopping Finance My portfolio My watchlist News Stock market Economics Earnings Crypto Politics Biden economy Personal finance Markets Stocks: most active Stocks: gainers Stocks: losers Trending tickers Futures World indices US Treasury bonds Currencies Crypto Top ETFs Top mutual funds Highest open interest Highest implied volatility Currency converter Sectors Basic materials Communication services Consumer cyclical Consumer defensive Energy Financial services Healthcare Industrials Real estate Technology Utilities Screeners Watchlists Equities ETFs Futures Index Mutual funds Analyst rating screener Technical events screener Smart money screener Top holdings screener Personal finance Credit cards Balance transfer cards Cash back cards Rewards cards Travel cards Banking CD rates Online checking High-yield savings Money market Mortgage Home equity loan HELOC Personal loans Student loans Insurance Taxes Videos ETF report FA corner Options pit Crypto Industries Sports Fantasy News Fantasy football Best Ball Pro Pick 'Em College Pick 'Em Fantasy baseball Fantasy hockey Fantasy basketball Download the app Daily fantasy NFL News Scores and schedules Standings Stats Teams Players Drafts Injuries Odds Super Bowl GameChannel Videos MLB News Scores and schedules Standings Stats Teams Players Odds Videos World Baseball Classic NBA News Draft Scores and schedules Standings Stats Teams Players Injuries Videos Odds Playoffs NHL News Scores and schedules Standings Stats Teams Players Odds Playoffs Soccer News Scores and schedules Premier League MLS NWSL Liga MX CONCACAF League Champions League La Liga Serie A Bundesliga Ligue 1 World Cup College football News Scores and schedules Standings Rankings Stats Teams Show all MMA WNBA Sportsbook NCAAF Tennis Golf NASCAR NCAAB NCAAW Boxing USFL Cycling Motorsports Olympics Horse racing GameChannel Rivals Newsletters Podcasts Videos RSS Jobs Help World Cup More news New on Yahoo Games Tech Terms Privacy Privacy & Cookie Settings Feedback US English Select edition USEnglish US y LATAMEspañol AustraliaEnglish CanadaEnglish CanadaFrançais DeutschlandDeutsch FranceFrançais 香港繁中 MalaysiaEnglish New ZealandEnglish SingaporeEnglish 台灣繁中 UKEnglish © 2024 All rights reserved. About our ads Advertising Careers Yahoo Home Yahoo Home Search query Select edition USEnglish US y LATAMEspañol AustraliaEnglish CanadaEnglish CanadaFrançais DeutschlandDeutsch FranceFrançais 香港繁中 MalaysiaEnglish New ZealandEnglish SingaporeEnglish 台灣繁中 UKEnglish News Finance Sports More News Today's news US Politics World Weather Climate change Health Science 2024 election Originals Life Health Parenting Style and beauty Horoscopes Shopping Food Travel Autos Gift ideas Buying guides Entertainment Celebrity TV Movies Music How to watch Interviews Videos Shopping Finance My portfolio Watchlists Markets News Videos Screeners Personal finance Crypto Industries Sports Fantasy NFL NBA MLB NHL College football College basketball Soccer MMA Yahoo Sports AM New on Yahoo Games Tech Selected edition USEnglish Mail Sign in Mail News Finance Sports Entertainment Life Yahoo Plus More... … Skip navigation linksSkip to main contentSkip to sidebarAdvertisementtop storiesUkraine warns situation on northern front has 'significantly worsened'The precise goal of Russia's new push in the northern Kharkiv region — which began in the early hours of Friday morning — is unclear. 'Ongoing battles in the border areas' »What is 'spaving' and why should buyers beware?Some Duke students walk out on Jerry Seinfeld's commencement speechTuna Crabs, neither tuna nor crabs, are swarming near San Diego'Completely different player': Djokovic 'concerned' about water bottle injuryAmericans are choking on surging fast-food pricesStories for you Sports·Yahoo SportsFormer NBA player Glen Davis sentenced to 40 months in prison for involvement in healthcare fraud schemeFormer NBA player Glen Davis was sentenced to 40 months in prison after his conviction for being involved in a scheme to defraud the league's healthcare plan.2 min read Thanks for your feedback! Sports·ProFootball Talk on NBC SportsNFL Schedule Release 2024: Primetime games, Thanksgiving and Christmas Day Schedule, International GamesAll the latest information on schedule and matchups for the 2024 NFL season.1 min read Thanks for your feedback! Technology·EngadgetHow to watch Google's I/O 2024 keynoteIt’s time for Google’s I/O 2024 conference. This event will spotlight Android 15 and Gemini, with room for plenty of surprises.2 min read Thanks for your feedback! Technology·Yahoo TechThe best noise-cancelling earbuds for 2024, tested and reviewedQuiet, cruel world! Drown the sound with our picks from Bose, Apple, Sony and more.6 min read Thanks for your feedback! Style·Yahoo Life ShoppingThe 9 best water shoes for women in 2024From rugged options for rocky beaches to cute, quick-drying espadrilles, these are the best women's water shoes, according to experts.5 min read Thanks for your feedback! Lifestyle·Yahoo Life Shopping27 last-minute Mother's Day gifts to shop nowIt's not too late! Score free, fast shipping on these last-minute gifts for Mom.2 min read Thanks for your feedback! NewsMotional delays commercial robotaxi plans amid restructuringMotional, the autonomous vehicle startup borne out of a $4 billion joint venture between Hyundai and automotive supplier Aptiv, will pause its commercial operations and delay plans to launch a driverless taxi service as it undergoes a restructuring, TechCrunch has learned. The aim is make progress on the core technology and the business model, while preserving capital, according to sources familiar with the changes. Thanks for your feedback! NewsApple’s Final Cut Camera lets filmmakers connect four cameras at onceThe latest version of Final Cut Pro introduces a new feature to speed up your shoot: Live Multicam. It’s a bold move from Apple, transforming your iPad into a multicam production studio, enabling creatives to connect and preview up to four cameras all at once, al in one place. From the command post, directors can remotely direct each video angle and dial in exposure, white balance, focus, and more, all within the Final Cut Camera app. Thanks for your feedback! NewsApple unveils the new 11-inch and 13-inch iPad Air with M2 chipApple is refreshing its iPad lineup today. And this launch is a bit overdue as the previous iPad Air was announced more than two years ago. The most important addition to the new iPad Air is that it now comes in two sizes. Customers can choose between an 11-inch display and a 13-inch display. Thanks for your feedback! NewsHere’s everything Apple just announced at its Let Loose event, including new iPad Pro with M4 chip, iPad Air, Apple Pencil and moreToday is Apple iPad Event day, and we’re ready to bring you all the iPad goodness you can stand, including if some of the rumors are true of what’s coming, like a new iPad Pro, iPad Air, Apple Pencil and a keyboard case. Don’t have time to watch? That’s ok — we’ve summed up the most important parts of the event below. Thanks for your feedback! NewsApple finally places front camera on iPad on the landscape edge — like it should beApple announced new iPads — two iPad Air models and the iPad Pro with M4 processor — at its “Let Loose” event on Tuesday. However, one of the important updates for people who take a lot of video calls was Apple shifting the camera to the landscape edge. Thanks for your feedback! Lifestyle·MoneywiseDave Ramsey says all Americans have ‘a shot’ at being millionaires — but people in 5 professions stand the best chanceThose who say it can’t be done … are “evil”4 min read Thanks for your feedback! Sports·Yahoo SportsNBA Draft Lottery: Hawks get No. 1 pick, despite 3 percent chance of winningThe Atlanta Hawks won the No. 1 overall selection in the NBA Draft Lottery. The Hawks had a 3 percent chance of winning the top pick.4 min read Thanks for your feedback! Sports·Yahoo Sports2024 NBA Mock Draft 7.0: Who will the Hawks take at No. 1? Our projections for every pick with lottery order now setWith the lottery order set, here's a look at Yahoo Sports' projections for both rounds of the 2024 NBA Draft.16 min read Thanks for your feedback! Business·Yahoo FinanceUtility stocks are on fire — here are Wall Street analysts' top picksUtility stocks are outperforming the broader markets. Here's a look at three top picks from analysts.5 min read Thanks for your feedback! Sports·Yahoo SportsDerrick Lewis strips off shorts, moons crowd in St. Louis after KO win over Rodrigo Nascimento“I appreciate St. Louis for letting me show my naked ass tonight."3 min read Thanks for your feedback! Sports·Yahoo SportsUFC St. Louis: Esteban Ribovics delivers wild head-kick KO to beat Terrance McKinney seconds into their fightIt took Esteban Ribovics only 37 seconds to get the win Saturday night.2 min read Thanks for your feedback! Business·Yahoo FinanceAustralian ambassador: 'American model is proving its resilience' despite threat from Chinese industrial policyChina may be outspending the US when it comes to industrial policy in sectors like electric vehicles and semiconductors, but America is winning on innovation where it can’t on price, according to one China expert.4 min read Thanks for your feedback! Business·Yahoo FinanceHere's 1 big investing mistake you are probably still makingMaybe a 5% CD isn't the best choice for your hard-earned money.3 min read Thanks for your feedback! Sports·Yahoo SportsRing announcer mistakingly names wrong winner of Cherneka Johnson-Nina Hughes bout in wild sceneLike Steve Harvey and the "Moonlight" debacle, Lt. Dan Hennessey made a brutal mistake on the mic on Sunday in Perth.2 min read Thanks for your feedback! Business·Yahoo FinanceFed officials stick to Powell’s higher-for-longer script as a key inflation reading loomsJay Powell’s colleagues spent the last week backing a stance the central bank chair hammered home at his last press conference: Interest rates will be staying higher for longer.5 min read Thanks for your feedback! Celebrity·Yahoo NewsWhy are social media users blocking celebrities and influencers on TikTok? A look at the #Blockout movementHere's why some social media users are blocking stars from their feeds after the Met Gala.3 min read Thanks for your feedback! Sports·Yahoo SportsNBA playoffs: Jayson Tatum, Celtics hold off late Cavaliers rally to take Game 3The Celtics now hold a 2-1 lead over the Cavaliers in their Eastern Conference semifinals series.3 min read Thanks for your feedback! Sports·Yahoo SportsNBA playoffs: P.J. Washington shines again as Mavericks hold off Thunder in Game 3, take 2-1 series leadP.J. Washington has scored 57 combined points in the last two games to lead Dallas past the Thunder.3 min read Thanks for your feedback! Sports·OUInsiderRed River Revenge: Sooners handle Texas, secure Big 12 ChampionshipThe Longhorns won in Austin, but the Sooners won the rematch for the Big 12 Championship on Saturday.5 min read Thanks for your feedback! Politics·Yahoo NewsBlinken, Sanders and Graham offer differing takes on latest State Dept. report on Israel's possible humanitarian violations in GazaAntony Blinken and Bernie Sanders praise Biden for withholding military aid to Israel. Lindsey Graham calls it the 'worst decision.'5 min read Thanks for your feedback! Business·Yahoo FinanceIs Austin’s hot housing market flipping in favor of buyers?The average home price in Austin, Texas, jumped by $170,000 during the pandemic. But the market is shifting in favor of buyers, and prices are beginning to come down.8 min read Thanks for your feedback! Lifestyle·Yahoo Life ShoppingThe 27 best Walmart deals to shop this weekend — save more than 80% on gardening supplies, tech items and moreSome major deals on board: A lovely vertical garden for just $40, a remarkable mini movie projector for a staggering $620 off and a handy Bissell upright vacuum for under $60.1 min read Thanks for your feedback! Entertainment·Yahoo TVSNL Recap: Maya Rudolph vogues her way back into 30 Rock in a Mother's Day-themed episodeThe 'SNL' alum revived her Beyoncé impression, with a new Cowboy Carter twist.4 min read Thanks for your feedback! Business·TechCrunchA reckoning is coming for emerging venture funds, and that, VCs say, is a good thingThousands of new venture capital funds have launched over the past few years, each hoping to carve out a long-term, lucrative place for themselves. PitchBook is tracking over 10,000 funds currently trying to raise money, and 45% of them are emerging fund managers, defined as a firm with less than three funds. For the most part, things seem to be shaking out quite nicely for emerging managers despite the economic headwinds.11 min read Thanks for your feedback! Lifestyle·Yahoo Life Shopping'None of my plants have died from overwatering since I started using this': This soil meter is on sale for just $14Make your garden the talk of the town with a little help from this friend.3 min read Thanks for your feedback! Lifestyle·Yahoo Life ShoppingThe T-shirt folding board shoppers say makes 'doing the laundry fun' is just $11 at AmazonThis problem-solving gadget takes the hassle out of getting your drawers and closets organized.3 min read Thanks for your feedback! Sports·Inside NebraskaBREAKING: Four-star QB TJ Lateef commits to NebraskaNebraska has landed their quarterback in 2025 in Orange (Cali.) Lutheran four-star TJ Lateef. Lateef committed to the Huskers two weeks after making an official visit to Lincoln for the Red-White Game.3 min read Thanks for your feedback! Business·Yahoo FinanceInflation and consumer spending updates ahead: What to know this weekA crucial reading of inflation faces investors in the week ahead after a string of disappointing reports to start the year dampened investors' interest rate cut hopes.7 min read Thanks for your feedback! Sports·Boiler UploadPurdue Baseball Drops Series at MichiganFor the second week in a row Purdue was able to win the Friday night opener of a key Big Ten series, but it also dropped the next two games to fall off the pace for the postseason. On Saturday Michigan beat Purdue 7-6 in walk-off fashion. On Sunday the Wolverines hit four home runs, three in the sixth inning, to take an 8-6 win and the series from the Boilers.5 min read Thanks for your feedback! Sports·OUInsiderOU sweeps Baylor, claims first regular season Big 12 title47. That is how may runs the Sooners' offense was responsible for putting on the Baylor Bears in the series this weekend2 min read Thanks for your feedback! Sports·Inside ND SportsNotre Dame Football Transfer Tracker for 2023-24 offseasonTracking Notre Dame football's outgoing transfers hitting the portal and incoming commitments from other schools1 min read Thanks for your feedback! Business·Yahoo FinancePrivate equity is helping banks shed some of their risksBanks are using "credit risk transfers" with private equity firms to help alleviate their regulatory burdens and protect against future losses. Thanks for your feedback! Sports·RivalsFact or Fiction: Five-star Solomon Thomas may flip to FloridaFive-star offensive lineman Solomon Thomas committed to Florida State in December.6 min read Thanks for your feedback! Lifestyle·Yahoo Life Shopping'Transformed my grilling experience': This No. 1 bestselling grill basket is just $26A barbecue enthusiast's dream, this gadget comes with a brush and heat-resistant glove.3 min read Thanks for your feedback! Sports·Yahoo SportsPirates pitching phenom Paul Skenes lives up to the hype in wild, rain-delayed debutThe Pirates topped the Cubs 10-9 in a game that offered a reminder that Skenes alone cannot save Pittsburgh.7 min read Thanks for your feedback! Lifestyle·Yahoo Life ShoppingThese 10 trending cleaning products — all under $20 — have thousands of five-star Amazon ratingsThe Pink Stuff, Scrub Daddy and more: Scour your space from top to bottom with essentials starting at just $5.4 min read Thanks for your feedback! Politics·HuffPostSpringsteen Fans Roast Trump For Bizarre Boast About Size Of New Jersey CrowdA spokesperson for the city of Wildwood said Trump spoke to 80,000 to 100,000 people, but video of the event made that hard to believe.3 min read Thanks for your feedback! Politics·AFPTrump, reciting songs and praising cannibals, draws yawns and raises eyebrowsFormer US president Donald Trump raised eyebrows among his no-nonsense New Jersey crowd at a rally Saturday when he recited a 1963 soul song in its entirety and referenced a fictional cannibal as "great.""A tenderhearted woman saw a poor, half-frozen snake," Trump said, reciting -- but not singing -- the song to the crowd.2 min read Thanks for your feedback! Politics·Fox NewsMassive Trump beach rally in deep blue NJ draws stark contrast to Biden's beach weekend: 'Biden could never'Critics on social media argue President Biden could not pull a massive crowd like former President Trump in Wildwood, New Jersey, as they slam the current president for his beach weekend.7 min read Thanks for your feedback! US·CNNA 19-year-old called 911 in a crisis and ended up dead. For advocates, it’s a call for a better way of policing“Please don’t shoot,” Notan Eva Costa, a 48-year-old Bangladeshi immigrant living in Queens, begged the two New York City police officers who entered her home after her teenage son called 911.9 min read Thanks for your feedback! Celebrity·SheKnowsInsiders Claim Kelly Preston’s Death Made John Travolta Do a 180 on This Part of His Life With ScientologyIt’s no secret that after Kelly Preston died in 2020, John Travolta has been rather low-key from the public eye. Despite a few appearances with their kids and an alleged romance blooming between him and Kristin Davis, Travolta has been quite private, and apparently, a lot has changed behind closed doors. An insider told Closer …2 min read Thanks for your feedback! US·NY PostThe city told a homeowner to hide his boat behind a fence — his creative and hilarious response went viralIs this guy the B.O.A.T., or what?2 min read Thanks for your feedback! Politics·SheKnowsA Super-Rare Clip of Barron Trump’s Voice Has the Internet Comparing Him to This Family MemberWhile Barron Trump may not be entering politics like his dad, former US President Donald Trump, and his siblings, he has something in common with his dad: their voices. Let us explain. An account called @realcherokeeowl shared a video clip of Barron at Mar-a-Lago, and it’s going viral on X, the app formerly known as …2 min read Thanks for your feedback! Celebrity·Miami HeraldDad reveals the last words his wife ever said. They were for their 4-year-old sonTaylor Odlozil opened up about his wife’s final moments after an 8-year battle with cancer and what she said to their son.2 min read Thanks for your feedback! Celebrity·SheKnowsKatie Holmes’s Daughter Suri Just Ensured Her Father Tom Will Have No Part in Her Theater CareerWith two parents who act, it’s no wonder that Katie Holmes and Tom Cruise’s daughter Suri Cruise got the acting bug. Not only did she star in her high school’s musical of Addams Family, which onlookers said she was “amazing” at, but she’s acting in local theater too! Along with that, eagle-eyed fans noticed a …2 min read Thanks for your feedback! Celebrity·PeopleMeghan Markle Wears Traditional Skirt Gifted to Her in Nigeria After Admitting She Needs to 'Wear More Color'The Duke and Duchess of Sussex have stepped out for their last day in Nigeria5 min read Thanks for your feedback! Celebrity·VarietyJennifer Lawrence Slams Mike Pence at GLAAD Media Awards: ‘Conversion Therapy Isn’t Real,’ Even Though ‘You Think It Worked on You’Ross Matthews wasn’t three words into his opening monologue at the 35th annual GLAAD Awards in New York City Saturday evening when a protester stood up from the audience to shout, “GLAAD is complicit in genocide.” The lone pro-Palestine advocate, blanketed by a wide-eyed room who let out a few jeers — “Who are you, you …5 min read Thanks for your feedback! Celebrity·PeoplePhoebe Cates and Kevin Kline's 2 Kids: All About Owen and Greta KlinePhoebe Cates and Kevin Kline share a son and daughter6 min read Thanks for your feedback! Celebrity·Fox NewsHugh Jackman's ex-wife Deborra-lee Furness learned she's 'strong and resilient' after splitHugh Jackman's ex-wife Deborra-lee Furness opened up about what she learned during her "year of evolution" following their split after 27 years together.3 min read Thanks for your feedback! Celebrity·SheKnowsJohn Travolta’s Alleged New Romance May Have a Major Hurdle on His EndWhile the internet and friends of the two are shipping John Travolta and Kristin Davis together, their alleged romance could have a huge hurdle on Travolta’s side. Insiders said to Closer Weekly on May 8 that Travolta is having a hard time moving on from the passing of his late wife, Kelly Preston. Now, this …2 min read Thanks for your feedback! Celebrity·PeopleWoman Learns She’s Expecting 'Perfect' Rainbow Baby, Then Finds Out She Has Breast Cancer (Exclusive)"We're not the type of people to just lay down and cry," Tiffany Gagliardi, whose son is due in July, tells PEOPLE6 min read Thanks for your feedback! Celebrity·SheKnowsNicole Kidman Reportedly Continues to Be Estranged From One of Her Children, Sources ClaimNicole Kidman is a loving mother of four, and while two of her children are incredibly close with their father, her ex-husband, Tom Cruise, they reportedly have quite a different relationship with their mom. While her adopted daughter Bella is allegedly still on good terms with her, her adopted son Connor is reportedly estranged from …1 min read Thanks for your feedback! Celebrity·PeopleAnnaLynne McCord Jokes Her 'Clothes Evaporate' Around Boyfriend She 'Reconnected' with After Years Apart (Exclusive)The '90210' alum is dating British rugby player Danny Cipriani, after the pair first met on the dating app Raya many years ago3 min read Thanks for your feedback! World·Fox NewsIranian lawmaker declares Tehran obtained nuclear bombsU.S. and UN experts say Iran’s regime is dangerously close to becoming a nuclear-armed weapons state as officials in Tehran announce their capability to build atomic bombs.6 min read Thanks for your feedback! Politics·Business InsiderTrump leads his supporters in vulgar chants at Jersey Shore rallyDonald Trump attacked his hush money trial prosecutor and judge at a New Jersey rally. He claimed he was being indicted on "bullshit" charges.3 min read Thanks for your feedback! US·NBC NewsTrump, gun owners and Jan. 6 rioters: Tough-on-crime Justice Alito displays empathy for some criminal defendantsConservative Justice Samuel Alito, a former U.S. attorney with a long history of voting in favor of prosecutors, has shown signs of empathy for defendants.5 min read Thanks for your feedback! Business·Motley FoolFact or Fiction: Congress Stole From Social Security and Should Return the Money It's Taken, With InterestDid lawmakers really pilfer Social Security's piggy bank? Here's the truth.8 min read Thanks for your feedback! US·Akron Beacon JournalFox 8's Wayne Dawson recuperating from surgery, thanks viewers for well-wishesIn April, Fox 8's Wayne Dawson underwent a lengthy surgery at the Cleveland Clinic. According to a message posted Sunday, he continues to recover.1 min read Thanks for your feedback! US·Fox NewsMassachusetts man dies 2 months after becoming first person to receive successful pig kidney transplantA Massachusetts man who was the first recipient of a genetically modified pig kidney transplant died nearly two months after he underwent the procedure.2 min read Thanks for your feedback! Politics·The HillVance says he was ‘wrong’ in past remarks on TrumpSen. JD Vance (R-Ohio) said Sunday that his past remarks about then-candidate Donald Trump ahead of the 2016 election were “wrong.” CNN’s Dana Bash pressed Vance on a tweet that he posted in 2016 when the “Access Hollywood” tape of Trump surfaced that stated, “Fellow Christians, everyone is watching us. When we apologize for this…2 min read Thanks for your feedback! Politics·The IndependentPotential VP pick joins Trump on flight to raucous Jersey Shore rally after he nixed another hopefulMr Trump tells one former rival to ‘get ready’ and another to get lost as he stokes speculation around vice president pick3 min read Thanks for your feedback! Sports·MMA JunkieWorst leg break in MMA history? Be the judge, if you have the stomach to watchAnderson Silva, Conor McGregor and Chris Weidman move aside – this might be the most gruesome leg break in MMA history.1 min read Thanks for your feedback! Lifestyle·Tasting TableThe Store-Bought Tortilla Brand You Should Keep Out Of Your Grocery CartNot all flour tortillas are the same, as our taste testers found out. There's one brand that's best avoided if you want a tasty meal that holds together.3 min read Thanks for your feedback! US·WPIX New York City, NYMap: Will you see the northern lights again on Sunday?More "very fast moving coronal mass ejections" are expected to hit Earth's magnetic field on Sunday — here's how it impacts your odds of seeing the northern lights.2 min read Thanks for your feedback! US·WFXTWeymouth man who received first-of-its-kind pig kidney transplant has diedThe Weymouth man who became the first person to receive a genetically edited pig kidney in March has died, Mass General Hospital says.3 min read Thanks for your feedback! Sports·CNNLionel Messi appears unhappy with new MLS rule as he is forced to wait on sidelines before returning to the pitchLionel Messi appeared to cut a frustrated figure on the sidelines when a new Major League Soccer rule prevented him from immediately returning to the pitch following an injury check.2 min read Thanks for your feedback! World·Fox NewsHere's why Ireland is at boiling point over mass immigrationHundreds of protests in towns have sprung up in Ireland with people calling on the government to end what they say is an "open borders" globalist agenda that is putting the needs of migrants ahead of its citizens.11 min read Thanks for your feedback! Lifestyle·Daily Meal9 Chain Restaurants That Use The Highest-Quality SeafoodChain restaurants aren't necessarily known for quality. But these 9 chains distinguish themselves by serving the highest-quality seafood possible.10 min read Thanks for your feedback! Celebrity·BuzzFeedEmployees Who've Had Or Witnessed Scandalous Workplace Affairs Are Telling Their Stories, And Ohhh Boyyyy"The company covered the whole thing up and blamed me. I quit before hitting my year mark."14 min read Thanks for your feedback! US·Patriot LedgerWeymouth man who had experimental pig kidney transplant has diedRichard Slayman, of Weymouth, had the surgery at Massachusetts General Hospital less than two months ago. When the 62-year-old left Mass General he said he had “one of the cleanest bills of health I’ve had in a long time.”2 min read Thanks for your feedback! Entertainment·Digital Trends5 sci-fi movies on Netflix you need to watch in May 2024Here are five sci-fi movies you need to watch on Netflix in May, including the sequel to an original space opera and an innovative action thriller.4 min read Thanks for your feedback! Sports·Steelers WireFormer Bears coach says Steelers QB Justin Fields was ‘hard to watch’One former Bears assistant was not impressed by Justin Fields.1 min read Thanks for your feedback! Entertainment·Louder Sound“Steve Winwood has his failings… I guess you can’t have everything”: The story of TrafficIt seems every member of the groundbreaking band paid a price for their involvement – but a three-album run of rare quality remains as testimony to their lofty creative intentions17 min read Thanks for your feedback! NextTrending Now1.NBA Draft Lottery2.Donald Trump3.Aurora Borealis4.Taylor Swift5.NASCAR Cup Series6.St. Louis Cardinals7.Eurovision8.Russia-Ukraine War9.Meghan Markle10.Minnesota VikingsAdvertisementWeatherGdanskView your LocationsDetect my locationView your LocationsPlease enable location service for your browserEnter City or ZipcodeTodayClear. Winds variable. The overnight low will be 39 °F (3.9 °C).58°39°MonSunny today with a high of 63 °F (17.2 °C) and a low of 42 °F (5.6 °C).63°42°TuePartly cloudy today with a high of 66 °F (18.9 °C) and a low of 39 °F (3.9 °C).66°39°WedMostly sunny today with a high of 68 °F (20 °C) and a low of 48 °F (8.9 °C).68°48°See moreScoreboardChange Sports to display different scoresTrending MLB NBA NCAAB NCAAF NFL NHL YesterdayTodayTomorrowNY 473rd 7:43IND 80TEX 1Top 7thCOL 2CIN 3Top 4thSF 0See moreDaily HoroscopeChange your horoscope signAquariusAriesCancerCapricornGeminiLeoLibraPiscesSagittariusScorpioTaurusVirgoTaurusMay 12 -You're feeling more than a little attached to someone, and that may or may not be a good thing! It's a good time to reconsider your relationships and decide what to do about some of them. See moreAdvertisement ADVERTISEMENT
Ćwiczenie 3: Napisz program do pobrania tekstu ze strony Wydziału Matematyki i Informatyki. Pobierz cały tekst ze strony głównej a następnie wyszukaj na tej stronie wszystkich linków wewnętrznych i pobierz tekst ze stron wskazywanych przez te linki. Nie zagłębiaj się już dalej.
def scrape_text(soup):
# Delete script and style elements
for script in soup(["script", "style"]):
script.extract()
# Get text
text = soup.get_text()
return text
def scrape_wmi():
url = "https://wmi.amu.edu.pl/"
page = requests.get(url)
soup = BeautifulSoup(page.content, 'html.parser')
# Create list to store text elements
text_elements = []
text = scrape_text(soup)
text_elements.append(text)
# Find all links
links = soup.find_all('a')
# Get href from links
links = [link.get('href') for link in links]
# Get text from links
for link in links:
try:
page = requests.get(link)
soup = BeautifulSoup(page.content, 'html.parser')
text = scrape_text(soup)
text_elements.append(text)
except:
pass
return text_elementslen(scrape_wmi())109
Omówione wyżej techniki działają również bardzo dobrze dla zasobów słownikowych.
Ćwiczenie 4: Pobierz jak najwięcej słów w języku albańskim z serwisu glosbe.com.
Rozumiem, że w tym zadaniu nie można skorzystać z biblioteki zawierającej np słownik z polskimi wyrazami (zbyt trywialne) - dlatego skorzystam ze słów z poprzedniego zadania, użyje lematyzacji i przetłumaczę je za pomocą serwisu
import spacy
def scrape_shqip(limit: int = 10):
base_url = "https://glosbe.com/pl/sq/"
unique_polish_words = list(set(" ".join(scrape_wmi()).split()))
unique_polish_words = [word.lower() for word in unique_polish_words if word.isalpha()]
unique_polish_words = unique_polish_words[:limit]
print(f"Polish words to translate: {unique_polish_words}", end="\n\n\n")
nlp = spacy.load("pl_core_news_sm")
doc = nlp(" ".join(unique_polish_words))
albanian_translations = {}
for word in doc:
word = word.lemma_
url = f"{base_url}{word}"
page = requests.get(url)
soup = BeautifulSoup(page.content, 'html.parser')
try:
albanian_word = soup.find('li', {'data-element':'translation'})
albanian_word = albanian_word.find('h3')
albanian_word = albanian_word.get_text().strip()
print(f"{word} -> {albanian_word}")
albanian_translations[word] = albanian_word
except:
print(f"{word} -> None")
albanian_translations[word] = None
return albanian_translationsalbanian_translations = scrape_shqip(limit=50)Polish words to translate: ['samorządu', 'ilorazowe', 'dużym', 'filozofii', 'zmienione', 'załączonych', 'wynikami', 'kampusie', 'powołać', 'matematycznego', 'uz', 'ostatniej', 'stopień', 'podręczników', 'advancedbeststudentgrant', 'filmy', 'powodu', 'infrastruktura', 'sesja', 'javascritp', 'wykładowcę', 'koordynatorką', 'minimalnych', 'administracyjnych', 'symulowania', 'soil', 'indeks', 'mail', 'konfigurujemy', 'skutki', 'analizując', 'humboldt', 'zainteresowane', 'trzyletnich', 'otwartością', 'mający', 'doradczyzielony', 'ustalonej', 'przykładów', 'czajkowskiizabela', 'sławomir', 'różniczkowychseminarium', 'przygotowuje', 'zdobywali', 'wjazd', 'wspinaczkowych', 'usługę', 'danychna', 'zastępowane', 'rozproszonym'] samorząd -> None ilorazowy -> None duży -> madh filozofia -> filozofia zmienić -> ndryshoj załączony -> None wynik -> rezultat kampus -> None powołać -> None matematyczny -> None uz -> None ostatni -> vonë stopień -> gradë podręcznik -> libër shkollor advancedbeststudentgrant -> None film -> film powód -> arsye infrastruktura -> Infrastruktura sesja -> sesion Javascritp -> None wykładowca -> mësimdhënës koordynatorka -> None minimalny -> None administracyjny -> None symulować -> None soil -> None indeks -> indeks mail -> None konfiguriwać -> None skutek -> rezultat analizować -> analizoj humboldt -> None zainteresować -> None trzyletni -> None otwartość -> None mieć -> kam doradczyzielon -> None ustalić -> None przykład -> shembull czajkowskiizabel -> None Sławomir -> None różniczkowychseminarium -> None przygotowywać -> None zdobywać -> fitoj wjazd -> aderim wspinaczkowy -> None usługa -> shërbim danychno -> None zastępować -> None rozproszony -> None