firecrawl_data_extraction_pipeline.py
Python · 909 lines183 class Valves(BaseModel):184 FIRECRAWL_API_KEY: str = Field(default="", description="Firecrawl API key")185 DEFAULT_FORMAT: str = Field(default="markdown", description="Default format for content extraction")186 ONLY_MAIN_CONTENT: bool = Field(default=True, description="Extract only main content")187 WAIT_FOR: int = Field(default=5000, description="Wait time in milliseconds")188 ENABLE_WEB_SEARCH: bool = Field(default=False, description="Enable web search for additional context")189 IGNORE_SITEMAP: bool = Field(default=False, description="Ignore sitemap.xml when extracting data")190 INCLUDE_SUBDOMAINS: bool = Field(default=False, description="Include subdomains when extracting data")191 SHOW_SOURCES: bool = Field(default=False, description="Show sources in the extraction results")192 BLOCK_ADS: bool = Field(default=True, description="Block ads during extraction")193 REMOVE_BASE64_IMAGES: bool = Field(default=True, description="Remove base64 encoded images from content")194 MOBILE: bool = Field(default=False, description="Use mobile user agent") A pipeline for structured data extraction from websites using Firecrawl API
0
0 A pipeline for structured data extraction from websites using Firecrawl API
0
0 
