How does schema evolution work?
Share
Lost your password? Please enter your email address. You will receive a link and will create a new password via email.
Please briefly explain why you feel this question should be reported.
Please briefly explain why you feel this answer should be reported.
Please briefly explain why you feel this user should be reported.
Schemas have advanced to the point where the same set of data can be stored in numerous files with different but compatible schemas. You can automatically identify and combine those files’ schema by using Spark’s Parquet data source.
A typical approach to dealing with schema evolution without automatic schema merging is to reload historical data, which is time-consuming.